What is RAG and why do you need it to build your own LLM app?#llm #rag
RAG (Retrieval-Augmented Generation) is a technique to enrich any LLM with your private data. You can't ask ChatGPT to search for a document on your Google Drive with a content related to your mom's birthday. OpenAI has trained their models on public sources, so it can't give you an answer for that. This is why you need RAG.

How RAG works?
First you need to embed your private data (e.g. all Google Drive files). Then, when we want to search over our vector store, we take the search query, embed it as well, and perform some sort of "similarity" search to find the most similar embeddings to our query. The simplest similarity measure is cosine similarity — we measure the cosine of the angle between each pair of embeddings. Or you can just use prepared retrievers for that task. Finally, retrieved information is fed into the LLM as an additional context to your prompt.

How text Embeddings are created?#llm #nlp
Embeddings are created on a pretrained models. There're many techniques to train such models (e.g. SBERT, RoBERTa), and they can be very specialized for a specific task, such as: search, context relevance, anomaly detection, visualization. Different models yield different vectors for the same sentence as they're trained for specific sentence contexts.

Where I can get pretrained models?
These models are obtained from such providers like OpenAI, Google PaLM, Hugging Face, etc. Usually, you don't need to download the heavy model to embed your text. For example, big players like OpenAI offer a pay-walled API for their text\-embedding\-ada\-002 embedding model.

Some models like all\-MiniLM\-L6\-v2 are very lightweight (80 MB) and can be used offline. This model will give you a 384-dimensional embedding vector (compared to 1536-dimensional vector of the OpenAI's state-of-the-art model).

What are Embeddings?#llm
An embedding is a numerical multidimensional vector which is created from a text or a blob (image, video, audio, etc.). It's created by AI to associate pieces of text or blobs with a more compact and easily comparable data structure which is a vector.

How do you use them?
Let's consider this example: some e-commerce store wants to power their search engine with AI capabilities:
1. Company creates embeddings for all their products based on descriptions
2. These embeddings are stored in an embedding space where millions of vectors associated with millions of products
3. User types a search query — "White Jordan Sneakers"
4. It's converted to an embedding vector — [0.1, 0.3, 0.45]
5. This embedding is compared to all vectors in an embedding space and backend returns N most similar vectors
6. Chosen vectors are converted back to the products by using the same embedding space
So in the end, the top result could be "Red Jordan Sneakers" with the embedding vector [0, 0.3, 0.67].

Project Loom Incubator#loom #concurrency #jep428
This is how common structured concurrency will look like:

```
try (var scope = StructuredTaskScope()) {
scope.fork(Weather::callApiMethod);
scope.join();
}

`` **What is the difference between ScopeObject and ExecutorService?**You create your executor during startup and shuts down when application is terminated.ExecutorServiceis using platform threads and they are 'expensive'. Scopes are just launchers for virtual threads, they don't need to store and pull them from a queue because they are cheap. Once you done withScopeObject` you can garbage it.

What Scheduler will Scopes use be default?Scopes use work-stealing approach based on async ForkJoinPool. Thus, every thread maintains a task deque and executes the task from its head. Furthermore, any idle thread does not block, waiting for the task and pulls it from the tail of another thread's deque instead.
https://www.youtube.com/watch?v=2nOj8MKHvmw

How to reduce Java Runtime footprint for cloud native applications?#runtime #cloud #containers
Remember the times when we had two implementations of the HotSpot JVM: client and server? Client was optimized for reduced startup time and memory footprint; and server was designed for max execution speed.

Today we have a tendency for custom JRE per project's need. You can use one version for CI/CD tests; second version for development; and third version for production environment.

How to make one?Do a static analysis of code dependancies with jdeps and remove unused JMPS modules with jlink. jlink tool is used to post-process JDK and create a smaller image by keeping only a specified set of modules and debug information.
https://docs.microsoft.com/en-us/java/openjdk/java-jlink-runtimes

Humongous Objects#g1 #gc
Humongous objects are very big objects (bigger than 1 MB) in heap that may cause performance degration of GCs.
• A humongous object is allocated directly in the Old Gen
• A separate humongous region is created for each such object by concatenating some number of contiguous regions
• Each such region can accommodate only one humongous object and nothing else

How objects are allocated in G1?By design, the G1 manages the heap by dividing it into a fixed number of same-size regions (depending on max heap size): 2MB, 4MB, 8MB, etc. Normally objects are allocated into a given region until it's full, and then at some point, the GC frees up the entire region by evacuating all live objects from it.

Why such objects are bad for GC?• If allocated in the Old Gen, they cannot be GCed quickly even if they are short-lived
• Merging regions into one might take time
• If there are many humongous objects on the heap, it can lead to heap fragmentation because of unused "gaps" in humongous regions.

A list of all JDK options in any implementation#jdk #configuration
Just a useful list of JDK Command Line Arguments. This list can be used for finer tuning of your production Java environments, or you can compare these options between versions when you do a migration to a newer JDK version.
https://chriswhocodes.com/

Pinpoint — Application Performance Management tool#performance #agent #apm
Pinpoint provides a solution to help analyze the overall structure of the system and how components within them are interconnected by tracing transactions across distributed applications. It is modeled on the tracing technique of Google’s Dapper but has been modified to add application-level tag data in the call header to trace distributed transactions at a remote call. The tag data consists of a collection of keys, which is defined as a TraceId.

You should definitely check Pinpoint out If you want to:• understand your application topology at a glance
• monitor your application in Real-Time
• gain code-level visibility to every transaction
• install APM Agents without changing a single line of code
• have minimal impact on the performance (approximately 3% increase in resource usage)
• use private cloud without out-of-the-box observability solution
https://pinpoint-apm.github.io/pinpoint/

What is a suppressed exception?#exception #jdk7
While browsing though the debris of exception's stack trace you might find such construction:

Caused by: java.io.FileNotFoundException: file.csv at io.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:123) ~[library.jar:12.345] ... 138 more Suppressed: java.lang.UnsatisfiedLinkError: ...

What does 'suppressed' keyword mean?It means that the original exception was suppressed/concealed by another exception. A common scenario for this is when the finally block throws an exception. Any exception originally thrown in the try block is then suppressed. Starting with Java 7, we can now use two methods on the Throwable class to handle our suppressed exceptions: addSuppressed and getSuppressed. They allow us to print all stack traces in the output stream regardless of where the exception occurred.
https://www.baeldung.com/java-suppressed-exceptions

Spring Framework RCE, Early Announcement#security #spring
RCE vulnerability was found in the Spring Framework that was leaked out ahead of CVE publication.

What is the issue?"In certain configurations, exploitation of this issue is straightforward, as it only requires an attacker to send a crafted HTTP request to a vulnerable system". The issue is simply evolved from a common principle "don't exec untrusted code in your webapp" which is a rule in every language.

Am I Impacted?Despite the public availability of PoC exploits, it's currently unclear which real-world applications use the vulnerable functionality. These are the requirements for the specific scenario from the report:
JDK 9 or higher:
• Apache Tomcat as the Servlet container
• Packaged as a traditional WAR (in contrast to a executable JAR)
• spring\-webmvc or spring\-webflux dependency
• Spring Framework versions 5.3.0 to 5.3.17, 5.2.0 to 5.2.19, and older versions
https://spring.io/blog/2022/03/31/spring-framework-rce-early-announcement