CatnipCoder

Graph-augmented retrieval and reciprocal rank fusion

Sat, 20 Jun 2026 00:00:00 +0300

Semantic search returns code that looks like what you asked about. Useful for conceptual questions, less so for everything else. The remaining questions split between exact-identifier lookup (you know the symbol name and want its definition) and structural traversal (you want callers, implementors, or the methods a change would touch). Each method handles its own question type well and handles the other two poorly. Running all three together and fusing their ranked outputs through reciprocal rank fusion covers the full range.

Three retrieval lanes

The previous article built a symbol-granular embedding index. That index handles natural-language questions where the identifier names are unknown: “how does the system handle expired sessions?” The embedding model maps query and chunks into the same space and returns nearest neighbours.

Two question types it handles poorly.

“Find the definition of findUserByEmail.” The embedding index returns code conceptually about finding users by email: related classes, helper methods, tests. Keyword search returns the symbol literally named findUserByEmail at rank 1.

“What calls validateSession?” The embedding index returns code that looks like it might call the method. The call graph returns every caller, exactly and completely. Top-K similarity is the wrong tool for a question whose answer is “all nodes with an outgoing CALLS edge to this method.”

Real queries mix these modes more than the categories suggest. A user describes a symbol by behaviour when they half-remember the name. A structural question arrives about something found through semantic search first. Running all three lanes and fusing the results handles that overlap.

BM25 for code

BM25 is the ranking function behind most keyword search systems. A document scores highly when it contains the query’s terms, especially rare ones, in a short body. The formula normalises for document length and term frequency so scores are comparable across the corpus.

For source code, one adjustment is needed: a tokeniser that understands identifier naming conventions. A standard whitespace tokeniser treats processOrderRefund as one opaque token. A query for Order should match it; a standard tokeniser makes no connection.

import re

CAMEL_BOUNDARY = re.compile(r"(?<!^)(?=[A-Z])")
NON_WORD = re.compile(r"[^A-Za-z0-9_]+")

def tokenise_code(text: str) -> list[str]:
    seen: set[str] = set()
    out: list[str] = []

    def emit(token: str) -> None:
        token = token.lower()
        if token and token not in seen:
            seen.add(token)
            out.append(token)

    for raw in NON_WORD.split(text):
        if not raw:
            continue
        emit(raw)
        is_mixed_case = any(c.isupper() for c in raw) and any(c.islower() for c in raw)
        if is_mixed_case:
            for part in CAMEL_BOUNDARY.split(raw):
                emit(part)
        if "_" in raw:
            for part in raw.split("_"):
                emit(part)
    return out

Applied to processOrderRefund, this emits the original token lowercased, then process, order, refund as CamelCase parts. Applied to MAX_RETRY_COUNT, it emits max_retry_count, then max, retry, count as snake_case parts. The all-uppercase guard prevents the CamelCase pass from running on screaming-snake constants, which would otherwise produce single-letter noise.

Deduplication through seen matters: identifiers combining both conventions produce overlapping splits without it, inflating the corpus term frequencies.

Build the index over the same chunks the embedding index uses, with the same IDs. The fusion step requires that all lanes refer to the same underlying symbol by the same identifier.

from rank_bm25 import BM25Okapi

tokenised_corpus = [tokenise_code(chunk.text) for chunk in chunks]
bm25 = BM25Okapi(tokenised_corpus)

def keyword_search(query: str, top_k: int = 50) -> list[str]:
    tokens = tokenise_code(query)
    scores = bm25.get_scores(tokens)
    ranked = sorted(
        zip([c.id for c in chunks], scores),
        key=lambda x: x[1],
        reverse=True,
    )
    return [chunk_id for chunk_id, score in ranked[:top_k] if score > 0]

rank_bm25 is adequate for codebases up to a few hundred thousand chunks. For larger collections, an on-disk inverted index (Whoosh, Tantivy, Elasticsearch) handles the scale without changing anything in the fusion layer.

The code graph

The code graph stores symbols as nodes and relationships as typed edges: CALLS, IMPLEMENTS, EXTENDS, USES. It is built from the same symbol extraction that produced the chunks in the previous article.

Querying it for a structural question is a depth-limited traversal from a seed node:

def graph_search(
    seed_fqn: str,
    edge_types: list[str],
    direction: str = "incoming",
    depth: int = 2,
    top_k: int = 50,
) -> list[str]:
    visited: set[str] = set()
    frontier = {seed_fqn}
    results: list[str] = []

    for _ in range(depth):
        next_frontier: set[str] = set()
        for node in frontier:
            neighbours = graph.neighbours(
                node, edge_types=edge_types, direction=direction
            )
            for neighbour in neighbours:
                if neighbour not in visited:
                    visited.add(neighbour)
                    results.append(neighbour)
                    next_frontier.add(neighbour)
        frontier = next_frontier
        if len(results) >= top_k:
            break

    return results[:top_k]

A query for “what calls validateSession?” runs graph_search("com.example.auth.validateSession", ["CALLS"], direction="incoming", depth=1) and returns the complete set of direct callers. Depth 2 returns callers-of-callers. The graph does not rank these; the fusion step handles ordering.

The graph’s answer is exact within the statically resolved call graph. Dynamic dispatch, reflection, and configuration-driven routing are outside what static analysis can see. For codebases that rely heavily on those patterns, the graph undercounts callers and keyword search on the method name provides a useful complement.

Reciprocal rank fusion

Three ranked lists arrive, scored on incompatible scales. Embedding similarity scores cluster between 0.4 and 0.9. BM25 scores spike or fall to zero with no upper bound. The graph produces an unordered set with no scores at all. Adding them up with weights does not work: whichever lane has the densest score cluster at the top dominates regardless of the weights.

Reciprocal rank fusion (RRF) ignores raw scores entirely. The fused score for a document d across a set of rankings is:

RRF(d) = Σ  1 / (k + rank_r(d))

where rank_r(d) is the position of d in ranking r, and k is a constant (default 60). If d does not appear in a ranking, that ranking contributes 0 to its sum.

A concrete example with three lanes and k=60:

Graph:     [A, B, C, D]
Embedding: [B, A, E, F]
Keyword:   [A, G, B, H]

Document A is at rank 1 in Graph, rank 2 in Embedding, rank 1 in Keyword:

RRF(A) = 1/(60+1) + 1/(60+2) + 1/(60+1) ≈ 0.0489

Document B is at rank 2, 1, 3:

RRF(B) = 1/(60+2) + 1/(60+1) + 1/(60+3) ≈ 0.0484

Document G appears only in Keyword at rank 2:

RRF(G) = 1/(60+2) ≈ 0.0161

A and B rise to the top because they appear across multiple lanes. G is buried despite ranking well in its single lane. Agreement across lanes is a stronger signal than dominance within one.

RRF is practical for four reasons: rank position is the only input, so no score normalisation is needed; a lane absent for a given document contributes 0 rather than a penalty; there is one tunable parameter (k) and the default of 60 works across most applications; and the sort is deterministic with a stable tie-break.

from collections import defaultdict

def reciprocal_rank_fusion(
    rankings: list[list[str]],
    k: int = 60,
    top_k: int = 50,
) -> list[tuple[str, float]]:
    fused_scores: dict[str, float] = defaultdict(float)
    first_seen: dict[str, int] = {}
    counter = 0

    for ranking in rankings:
        for rank, doc_id in enumerate(ranking, start=1):
            fused_scores[doc_id] += 1.0 / (k + rank)
            if doc_id not in first_seen:
                first_seen[doc_id] = counter
                counter += 1

    return sorted(
        fused_scores.items(),
        key=lambda kv: (-kv[1], first_seen[kv[0]]),
    )[:top_k]

The first_seen map gives a deterministic tie-break: documents with identical fused scores are ordered by when they first appeared across the input rankings. Without this, the sort order depends on dict iteration order, which is an insertion-order detail you should not rely on.

Running lanes concurrently

The three lanes are independent and run concurrently. Each gets a per-lane timeout so a slow embedding service or a graph blip does not block the whole query.

import asyncio
import logging

log = logging.getLogger(__name__)
PER_LANE_TIMEOUT = 3.0

async def _safe(name: str, coro):
    try:
        return name, await asyncio.wait_for(coro, timeout=PER_LANE_TIMEOUT)
    except asyncio.TimeoutError:
        log.warning("lane %s timed out", name)
        return name, []
    except Exception:
        log.exception("lane %s failed", name)
        return name, []

async def hybrid_search(
    question: str,
    use_graph: bool = False,
    graph_seed: str | None = None,
    use_embedding: bool = True,
    use_keyword: bool = True,
) -> list[str]:
    tasks = []
    if use_embedding:
        tasks.append(_safe("embedding", embedding_search(question, top_k=50)))
    if use_keyword:
        tasks.append(_safe("keyword", keyword_search(question, top_k=50)))
    if use_graph and graph_seed:
        tasks.append(_safe("graph", graph_search(graph_seed, ["CALLS"], top_k=50)))

    results = dict(await asyncio.gather(*tasks))
    rankings = [ids for ids in results.values() if ids]
    return [doc_id for doc_id, _ in reciprocal_rank_fusion(rankings)]

The graph lane needs a seed node: a starting symbol to traverse from. The seed comes either from the question directly (the user named a symbol), from a planner that parsed an identifier out of the question text, or from the top result of a keyword pre-pass when the question described a symbol without naming it.

Which lanes to run

Not every question benefits from all three lanes. Running all three when only one is relevant adds latency and dilutes the fused result.

If the question contains an explicit identifier (backtick-quoted or recognisably formatted), run keyword and graph: embedding returns a vague superset and is not worth the latency. If the question is structural (“what calls”, “what implements”, “what changes if”), run the graph from the named seed and add keyword when dynamic dispatch could produce callers the graph cannot see. If the question is conceptual, in natural language with no specific identifier, run embedding plus keyword when the question contains domain vocabulary that maps to identifiers. For everything that mixes these shapes, run all three.

On a measured evaluation set, RRF fusion beats the best single lane by a small margin on pure-mode questions and by a larger margin on mixed-mode ones. Mixed-mode questions are more common than the categories suggest: real users phrase questions informally, and the phrasing rarely maps cleanly to a single retrieval strategy.

This is the third article in the Reliable by Design series. The previous article covered symbol-granular chunking. The next covers incremental graph updates: keeping the graph and indexes correct as the codebase changes, without rebuilding from scratch on every commit.

Symbol-granular chunking for code retrieval

Tue, 16 Jun 2026 00:00:00 +0300

Most RAG tutorials show you how to chunk documents: split on character count, add overlap, embed each chunk, store. For prose — docs, wikis, READMEs — this produces a working system. For source code it produces a system that looks like it works and retrieves the wrong thing regularly. This article covers why, what the right boundary is, and how to implement it using tree-sitter.

Why character-count chunking fails on code

The assumption behind fixed-size chunking is that any 1000-character window contains a roughly self-contained semantic unit. For prose that holds often enough. Code does not work this way.

A 1000-character window might contain half a method — a signature and the first 15 lines of a 30-line body. Or three short methods in their entirety, none of them dominant in the embedding. Or a method body with no signature, because the window started right after def.

Each of these produces a bad embedding. A chunk containing half a method body embeds as something between what the method does and what the surrounding code does. A chunk containing three unrelated short methods embeds as a blend of all three, and a query matching any one of them retrieves the other two at the same rank.

The failure is not loud. The system returns results, roughly in the right neighbourhood. The problem is that the correct result sits at rank 4 or rank 7 instead of rank 1, and across thousands of queries that is a meaningful drag.

Fixed-size overlap makes this worse. Overlap means the same method body appears as the tail of one chunk and the head of the next. A query that matches the method returns both in the top results. The user gets two copies of the same code in slots 1 and 2.

The unit that matters in code

The natural semantic unit in source code is the symbol: a function, a method, a class. Each has a fully qualified name that identifies it, a signature that describes its interface, and a body that implements its behaviour.

Chunking at symbol granularity means one chunk per symbol, with boundaries at the symbol’s actual start and end lines. Each retrieved chunk is a complete method or class. The embedding represents what that symbol does, not what an arbitrary window around it contains.

Deduplication also becomes trivial: the same symbol is always one chunk, not three overlapping copies.

Extracting symbols with tree-sitter

The cleanest way to find symbol boundaries is to parse the source and query the AST. Regex-based extraction misses method calls whose receiver is not a simple identifier, drops constructor invocations, and breaks on generated files with non-standard formatting. A real parser handles all of these.

tree-sitter is the practical choice: fast (tens of megabytes per second per core), error-tolerant (it recovers from syntax errors rather than stopping), and covering over 200 languages through separate grammar packages loaded at runtime. Adding a language is pip install tree-sitter-rust, not a fork.

from tree_sitter import Language, Parser, Query, QueryCursor
import tree_sitter_python as tspython

PY_LANGUAGE = Language(tspython.language())
parser = Parser(PY_LANGUAGE)

The query language lets you describe patterns as S-expressions rather than writing imperative tree-walking code:

function_query = Query(PY_LANGUAGE, """
(function_definition
  name: (identifier) @function.name
  parameters: (parameters) @function.params
) @function
""")

class_method_query = Query(PY_LANGUAGE, """
(class_definition
  name: (identifier) @class.name
  body: (block
    (function_definition
      name: (identifier) @method.name
      parameters: (parameters) @method.params
    ) @method)
) @class
""")

Running a query against a parsed file returns the matching nodes with their line positions:

source = open("payment_service.py", "rb").read()
tree = parser.parse(source)

cursor = QueryCursor(function_query)
matches = cursor.captures(tree.root_node)

for capture_name, nodes in matches.items():
    for node in nodes:
        if capture_name == "function":
            print(node.start_point, node.end_point)

The same pattern works across languages. TypeScript, Go, Scala, Java — each has its own grammar package and its own set of queries, but the output structure is identical. Everything downstream stays the same regardless of which language the source is in.

One operational note: tree-sitter’s Python bindings have shifted across versions. Pre-0.25, queries were constructed via Language.query() and called directly. From 0.25 onward, query execution is owned by QueryCursor. Pin the version in your pyproject.toml and run an extraction test against a known-good corpus on every upgrade. A binding change that silently drops a query type is harder to catch than a test failure.

Building the chunk

A symbol extracted from the AST gives you a line range. The chunk text is the source at that range, with a short header prepended:

def chunk_for_symbol(symbol, source_lines):
    body = "\n".join(source_lines[symbol.start_line:symbol.end_line + 1])
    header = f"# {symbol.kind} {symbol.fqn}\n"
    if symbol.enclosing_class:
        header += f"# in class {symbol.enclosing_class.fqn}\n"
    return Chunk(
        id=symbol.fqn,
        text=header + body,
        payload={
            "fqn": symbol.fqn,
            "kind": symbol.kind,
            "file": symbol.file,
            "line_range": [symbol.start_line, symbol.end_line],
            "language": symbol.language,
        },
    )

The header matters more than it looks. A short method body — three lines of arithmetic, a guard check, a return — carries almost no meaning in isolation. The embedding model has no idea what it is looking at. The header adds the fully qualified name and enclosing class, which is usually enough to disambiguate. PaymentService.refund and RefundCalculator.apply can have similar bodies; with the header, their embeddings differ.

The id is the symbol’s fully qualified name, not a random UUID. A stable ID means that when a method changes and gets re-indexed, the new vector overwrites the old one at the same position in the vector store. With unstable IDs, re-indexing appends a new row and the old vector keeps ranking against queries until a separate cleanup pass removes it.

For classes, a separate chunk captures the class’s overall structure rather than any single method:

def chunk_for_class(symbol, source_lines):
    header = f"# class {symbol.fqn}\n"
    if symbol.docstring:
        header += f"# {symbol.docstring.strip()[:200]}\n"
    signature = source_lines[symbol.start_line]
    method_summary = "\n".join(
        f"  {m.kind} {m.name}{m.signature}"
        for m in symbol.methods
    )
    return Chunk(
        id=f"{symbol.fqn}::class",
        text=f"{header}{signature}\n{method_summary}",
        payload={...},
    )

Queries about a class’s overall role retrieve the class-level chunk. Queries about a specific method retrieve the method-level chunk. The reranker decides which is more relevant for a given query.

Handling same-name symbols

The header-prefix approach has one failure mode: two methods named validate in different classes can still embed close together if their bodies are structurally similar. Both are guard checks that raise on bad input and return early. The header adds the class name, which helps, but when bodies are short the body still dominates.

The more complete fix is late chunking. Instead of encoding each chunk independently, encode the whole file in one pass and derive per-symbol vectors by pooling the token-level outputs that correspond to each symbol’s line range:

import torch

def late_chunked_vectors(model, file_text, chunk_token_ranges):
    tokenizer = model.tokenizer
    transformer = model[0].auto_model
    transformer.eval()

    encoded = tokenizer(
        file_text,
        return_tensors="pt",
        truncation=True,
        max_length=transformer.config.max_position_embeddings,
    )
    with torch.no_grad():
        token_embeddings = transformer(**encoded).last_hidden_state[0]

    chunk_vectors = []
    for start_tok, end_tok in chunk_token_ranges:
        pooled = token_embeddings[start_tok:end_tok].mean(dim=0)
        chunk_vectors.append(
            torch.nn.functional.normalize(pooled, dim=0).numpy()
        )
    return chunk_vectors

When the full file passes through the encoder as one sequence, each token’s output already incorporates context from the imports, the class signature, and the surrounding methods. The vector for LoginService.validate is built from encodings that have attended across login-flavoured context. The vector for PaymentRequest.validate has attended across payment-flavoured context. They end up far apart in embedding space.

The cost is indexing time. Encoding a full 800-token file is two to three times slower than encoding the chunk bodies independently. For batch indexing of a large codebase, that is hours of additional time. For incremental indexing on file changes it is negligible, since only the changed file is re-encoded.

Late chunking pays for itself when symbol names are reused across different contexts — validate, process, handle. When symbol names are already specific, the header-prefix approach is sufficient and simpler.

One thing to check before you ship

Embedding models for code are often asymmetric: stored chunks and incoming queries use different prefixes. Nomic Embed, for example, expects search_document: on stored chunks and search_query: on queries. Using the wrong prefix on either side shifts the embedding geometry and drops retrieval quality by several points with no obvious error.

Check the model’s documentation, and bake the correct prefix into both the chunker and the query path:

def embed_chunk(model, text):
    return model.encode(f"search_document: {text}")

def embed_query(model, query):
    return model.encode(f"search_query: {query}")

It is the smallest fix in this article and it affects every retrieval result.

This is the second article in the Reliable by Design series. The next article covers graph-augmented retrieval: building a code graph from the extracted symbols and combining it with embedding search and keyword search through reciprocal rank fusion.

Why LLM-powered tools fail in production

Sun, 14 Jun 2026 00:00:00 +0300

LLM-powered developer tools work in demos. They summarize a file, answer a question about a class, explain a function. Then they hit a real codebase — large, cross-repository, underdocumented — and the answers become wrong in ways that are hard to detect. This article explains why that happens and what a system that avoids it looks like.

The demo gap

A demo codebase fits in a context window. Production codebases don’t.

When an LLM receives the full relevant context for a question, it can answer correctly. When it receives a few retrieved chunks instead, it can only answer correctly if the retrieval step found the right ones. Most of the time it didn’t.

The structure problem is separate. Production codebases have methods that call other methods across packages, repositories, and generated code. The relationships between symbols aren’t visible from any single file. And the same name — validate, process, handle — might be defined in seventeen different packages. A retrieval step that returns “the most relevant code for validate” has no way to know which one the question is about.

The gap isn’t about model quality. It’s about what the model is given to work with.

Retrieval is the first failure mode

The standard architecture is retrieval-augmented generation: embed the codebase, embed the query, find the nearest neighbours, pass them to the model. This works for questions about what code looks like. It breaks for questions about what code does.

Similarity search finds files that mention the same terms, functions with similar names, comments that overlap with the question. What it can’t answer is: what calls this method? That answer is a traversal through a call graph, not a ranking by embedding distance.

A codebase that handles authentication has the complete answer to “what calls validateToken” in its call graph. No similarity engine extracts it from chunks. Building a code graph — symbols as nodes, calls and imports and inheritance as edges — is the only way to answer that class of question.

Without it, you get the definition of validateToken, the comments around it, the tests that call it. The twelve call sites spread across services? Not there.

Orchestration is the second failure mode

The typical approach to a complex question: gather everything that might be relevant, concatenate it, send it to the model, hope the context window holds.

It usually doesn’t. A question about a cross-service change needs multiple files, the relevant graph edges, test coverage data, schema definitions. That exceeds what a single call handles well, and the model starts losing coherence on earlier parts of the context by the time it works through the later ones.

What makes this worse is that not everything relevant is known upfront. Answering “is this change safe?” requires knowing what the change affects, which requires a graph traversal, which requires knowing the starting symbol, which may require a search step first. These are sequential dependencies. Stuffing them into a single prompt treats them as if they aren’t.

A better design separates planning, execution, and synthesis. A planner inspects the question and emits a structured list of tool calls — graph queries, searches, file reads — with their dependencies explicit. A dispatcher runs those calls, in parallel where it can, in sequence where it must. A synthesizer sees the results and produces the answer. The model handles planning and synthesis; execution is deterministic tool invocation.

Verification is the third failure mode

Even with good retrieval and structured orchestration, the model will still produce wrong answers. Not constantly, but often enough that you need a plan for it. The failure mode is confident incorrectness: a fluent, specific answer that is wrong in ways that don’t read as wrong. The model’s confidence is not a signal here — it’s equally confident about correct and incorrect answers.

Verification needs to be cheap enough to run on every response, because you don’t know in advance which ones are wrong. A two-stage approach works: a fast structural check first — does the answer reference symbols that actually exist, does it contradict the retrieved context — and a deeper per-claim pass only on responses the first stage flags.

The retrieval infrastructure is already there. An answer that claims method A calls method B can be checked against the call graph. An answer about test coverage can be checked against the coverage index. Verification is a structured comparison against facts the system already has.

What comes next

The next articles cover each of these layers with concrete implementations:

Symbol-granular chunking: splitting source code for retrieval without losing the structural unit the question is about
Graph-augmented retrieval: building the code graph and combining it with embeddings and keyword search through reciprocal rank fusion
Incremental graph updates: keeping the graph correct as the codebase changes, without rebuilding from scratch on every commit
Deterministic review rules: catching known-bad patterns before the LLM sees the code, with suppression rate tracking to detect rule decay
The plan/dispatch/synthesize pattern in detail
A two-stage verification cascade
Cost attribution by role: tracking which part of the system is responsible for API spend

Each article is self-contained. Read them in order for the full architecture, or jump to whichever component you’re building.

The art of slowing down: Why it makes you a better engineer

Tue, 05 Nov 2024 00:00:00 +0200

In this article I will talk about the importance of slowing down and how it can make you a better engineer.

Introduction

We all know that the tech world is moving rapidly, new innovations are surfacing every day and the competition is fierce. Every week there is a new javascript framework to learn and star on GitHub. Every day there is a new startup that is disrupting the industry and the word “AI” is thrown around like it’s the new buzzword.

It is easy to get caught up in the hustle and bustle of the tech world and feel like you’re falling behind. Like you’re never going to compete with the 10x engineers who’re working 80 hours a week and have a side project that is going to change the world. However, I believe that there is a better way to approach this problem. Instead of trying to keep up with the pace of the tech world, we should slow down and focus on the things that really matter.

The art of slowing down

Let’s face it, we all want to earn 10x more money, be 10x more productive, wear the hat of the superstar engineer and be the next Elon Musk. But realistically, only a tiny percentage of people achieve fame or massive success. Does that mean the rest of us are failing? Not at all.

Most of us are simply engineers who love what we do and are trying to make a good living. We don’t need to become the next Elon Musk or Mark Zuckerberg to be happy. What we need is to focus on what truly fulfills us in both our personal and professional lives.

But, how do we do that? How do we slow down in a world moving so fast? Here are a few tips that I’ve found helpful:

Take breaks: It is important to take breaks throughout the day to recharge your batteries. Working in a pomodoro style is an excellent way to keep up with the craziness of the day-to-day work.
Exercise : Exercise is a great way to clear your mind and reduce stress. It is also a great way to stay healthy and fit. There is nothing better than being a fit software engineer and being to run a race without needing to consume 10 cups of coffee and stay up late working on the next big thing.
Travel : You don’t need to travel to the end of the world or do expensive trips. Take a weekend off and go hiking or camping. It is a great way to disconnect from the world and recharge your batteries. You will be surprised that there is more in life than spending it in front of the computer screen.
Read books : Reading books is a great way to slow down and learn new things. It is also a great way to relax and escape from the craziness of the tech world. There is nothing better than reading a good book and sips a cup of coffee on a lazy Sunday afternoon.
Spend time with family and friends : Spending time with family and friends is a great way to slow down and appreciate the things that really matter in life. There is a famous quote that says “No one on their deathbed ever said, ‘I wish I had created more git commits and broke master more times’”. Spend time with your loved ones and enjoy the little moments in life because that moment never last forever.

Will Slowing Down Hurt My Career?

You might be thinking that slowing down will affect your career, and you will never be able to compete with the 10x engineers who’re working 80 hours a week. But here’s the thing: constantly running at full speed often leads to burnout and a loss of passion for your work.

Remember, you’re not a machine. Taking care of yourself physically, mentally, and emotionally makes you a better engineer in the long run. Sure, learn new things, work on side projects, but don’t sacrifice your health and well-being for career advancement.

Conclusion

Am I doing all of these things perfectly myself? Of course not! It’s a journey. But if this article helps me focus on what’s important and encourages you to do the same, then it’s worth it. Slowing down doesn’t make you a lesser engineer; it helps you be a better one, grounded in what matters.

Implementing a Validated Monad in Java and using it with Spring Boot

Fri, 16 Aug 2024 00:00:00 +0300

In this article we will implement a Validated monad in Java, explain its use cases and how it can be used with Spring Boot and any other Java framework.

For a convenient zero-dependencies library allowing you to start using useful functional programming constructs you can check out the FunctionalUtils library

For an introduction to functional programming you can read here.

Introduction

In functional programming, a monad is a design pattern that allows you to chain operations together in a way that is composable and more fluent to read. A Validated monad is a monad that allows you to chain operations together, but it also accumulates errors that occur during the operations. This is useful when you want to validate multiple things at once and return all the errors that occurred.

Implementing the Validated Monad

We will start by implementing the Validated monad. We will create a sealed interface Validated that will have two subclasses, Valid and Invalid. The Valid class will hold a value and the Invalid class will hold a list of errors.

To respect the properties of a monad, we will implement the unit and bind methods written as of and flatMap respectively. Additionally, we will implement a map method that will allow us to apply a function to the value of the Valid class as well as a mapN method that will allow us to combine multiple Validated instances and accumulate the errors.

public sealed interface Validated<E, A> permits Valid, Invalid {

  /**
   * Create a new Valid instance.
   *
   * @param value The value to wrap
   * @param <E>   The type of the error
   * @param <A>   The type of the value
   * @return A new Valid instance
   */
  static <E, A> Validated<E, A> valid(A value) {
    return new Valid<>(value);
  }

  /**
   * Create a new Invalid instance.
   *
   * @param error The error to wrap
   * @param <E>   The type of the error
   * @param <A>   The type of the value
   * @return A new Invalid instance
   */
  static <E, A> Validated<E, A> invalid(E error) {
    return new Invalid<>(error);
  }

  /**
   * Combines two or more {@code Validated} instances into a single instance. If all instances are
   * valid, the result is a valid instance with a list of values. If any instance is invalid, the
   * result is an invalid instance with a list of errors.
   *
   * @param validatedList a list of {@code Validated} instances
   * @param fn            a function that takes a list of values and returns a new value
   * @param <E>           the type of the error
   * @param <B>           the new type of the value
   * @return a new {@code Validated} instance
   */
  @SuppressWarnings("unchecked")
  static <E, B> Validated<List<E>, B> mapN(
      Function<List<Object>, B> fn, List<Validated<E, ?>> validatedList) {
    List<Object> values = new ArrayList<>();
    List<E> errors = new ArrayList<>();

    for (Validated<E, ?> validated : validatedList) {
      if (validated instanceof Valid) {
        values.add(((Valid<E, ?>) validated).value());
      } else {
        errors.add(((Invalid<E, ?>) validated).error());
      }
    }

    if (errors.isEmpty()) {
      return Validated.valid(fn.apply(values));
    } else {
      return Validated.invalid(errors);
    }
  }

  /**
   * Check if the instance is valid.
   *
   * @return true if the instance is valid, false otherwise
   */
  default boolean isValid() {
    return this instanceof Valid;
  }

  /**
   * Check if the instance is invalid.
   *
   * @return true if the instance is invalid, false otherwise
   */
  default boolean isInvalid() {
    return this instanceof Invalid;
  }

  /**
   * Get the error if the instance is invalid.
   *
   * @return The error
   */
  default E getInvalid() {
    return ((Invalid<E, A>) this).error();
  }

  /**
   * Get the value if the instance is valid.
   *
   * @return The value
   */
  default A getValid() {
    return ((Valid<E, A>) this).value();
  }

  /**
   * Map the value if the instance is valid.
   *
   * @param fn  The function to apply
   * @param <B> The new type of the value
   * @return A new Valid instance with the new value
   */
  default <B> Validated<E, B> map(Function<? super A, ? extends B> fn) {
    if (isInvalid()) {
      return Validated.invalid(getInvalid());
    }
    return Validated.valid(fn.apply(getValid()));
  }

  /**
   * FlatMap the value if the instance is valid.
   *
   * @param fn  The function to apply
   * @param <B> The new type of the error
   * @return A new Valid instance with the new value
   */
  default <B> Validated<E, B> flatMap(Function<? super A, ? extends Validated<E, B>> fn) {
    if (isInvalid()) {
      return Validated.invalid(getInvalid());
    }
    return fn.apply(getValid());
  }

  /**
   * Apply the first function if the instance is invalid, the second function if the instance is
   * valid.
   *
   * @param invalidFn the function to apply if the instance is invalid.
   * @param validFn   the function to apply if the instance is valid.
   * @param <L>       The new type of the error
   * @param <B>       The new type of the value
   * @return the result of applying the appropriate function.
   */
  default <L, B> Validated<L, B> bimap(Function<? super E, ? extends L> invalidFn,
      Function<? super A, ? extends B> validFn) {
    if (isInvalid()) {
      return Validated.invalid(invalidFn.apply(getInvalid()));
    }
    return Validated.valid(validFn.apply(getValid()));
  }

  /**
   * Applies a function to the value inside {@code Validated}, depending on whether the instance is
   * {@code Invalid} or {@code Valid}.
   *
   * @param invalidFn the function to apply if the instance is invalid.
   * @param validFn   the function to apply if the instance is valid.
   * @param <B>       The new type of the value
   * @return the result of applying the appropriate function.
   */
  default <B> B fold(Function<? super E, ? extends B> invalidFn,
      Function<? super A, ? extends B> validFn) {
    if (isInvalid()) {
      return invalidFn.apply(getInvalid());
    }
    return validFn.apply(getValid());
  }

  /**
   * Get the value if the instance is valid, or the other value if the instance is invalid.
   *
   * @param other The value to return if the instance is invalid.
   * @return The value if the instance is valid, or the other value if the instance is invalid.
   */
  default A getOrElse(A other) {
    if (isInvalid()) {
      return other;
    }
    return getValid();
  }

  /**
   * Swaps the sides of this {@code Validated}. If this is {@code Valid}, then the returned instance
   * will be {@code Invalid} with the same value and vice versa.
   *
   * @return A new instance with the value and the error swapped.
   */
  default Validated<A, E> swap() {
    if (isInvalid()) {
      return Validated.valid(getInvalid());
    }
    return Validated.invalid(getValid());
  }

  /**
   * Converts the instance to an {@code Optional}.
   *
   * @return An {@code Optional} instance.
   */
  default Optional<A> toOptional() {
    if (isInvalid()) {
      return Optional.empty();
    }
    return Optional.of(getValid());
  }
}

Example Usage of the Validated Monad

To apply the Validated monad, we will implement a sample Spring Boot application that demonstrates how to use the Validated monad to validate a form containing some person specific information and accumulate errors.

It consists of a REST endpoint that accepts a JSON payload containing some person identification information and validates it using the Validated monad. If there are any errors, it returns a 400 response with a JSON payload containing the errors. If the validation is successful, it returns a 204 response.

Setting up the Spring Boot Application

Let’s declare the dependencies required for the Spring Boot application in the pom.xml file.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.soumakis</groupId>
    <artifactId>Validated-Monad-Spring-Rest</artifactId>
    <version>1.0-SNAPSHOT</version>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>3.3.2</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>

    <properties>
        <maven.compiler.source>21</maven.compiler.source>
        <maven.compiler.target>21</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <functional.utils.version>2.2.0</functional.utils.version>
    </properties>

    <dependencies>
        <!-- Other dependencies -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-validation</artifactId>
        </dependency>
    </dependencies>

</project>

Implementing the Person DTO

We will start by implementing a DTO class that represents the person information.

package com.soumakis.example.domain;

public record PersonRequest(String name, Integer age, String city, String email, String password) {

}

Implementing the Person Validator

Next, we will implement a class that will validate the person’s information using the Validated monad we implemented earlier.

package com.soumakis.example.domain;
import org.springframework.stereotype.Service;

import java.util.List;
import java.util.Set;
import java.util.regex.Pattern;

@Service
public class PersonValidator {

    private static final Set<String> ALLOWED_CITIES = Set.of("New York", "Los Angeles", "Chicago",
        "Houston", "Phoenix", "Athens");
    private static final Pattern EMAIL_PATTERN = Pattern.compile(
        "^[A-Za-z0-9+_.-]+@[A-Za-z0-9.-]+$");
    private static final Pattern PASSWORD_PATTERN = Pattern.compile(
        "^(?=.*[0-9])(?=.*[!@#$%^&*])[A-Za-z0-9!@#$%^&*]{8,}$");
    private static final Pattern NAME_PATTERN = Pattern.compile("^[A-Za-z ]+$");

    public Validated<String, String> validateName(String name) {
        if (name == null || name.trim().isEmpty()) {
            return Validated.invalid("Name cannot be empty");
        } else if (!NAME_PATTERN.matcher(name).matches()) {
            return Validated.invalid("Name must not contain numbers or special characters");
        } else {
            return Validated.valid(name);
        }
    }

    public Validated<String, Integer> validateAge(Integer age) {
        if (age == null) {
            return Validated.invalid("Age is required");
        } else if (age < 18 || age > 100) {
            return Validated.invalid("Age must be between 18 and 100");
        } else {
            return Validated.valid(age);
        }
    }

    public Validated<String, String> validateCity(String city) {
        if (city == null || city.trim().isEmpty()) {
            return Validated.invalid("City cannot be empty");
        } else if (!ALLOWED_CITIES.contains(city)) {
            return Validated.invalid("City must be one of " + ALLOWED_CITIES);
        } else {
            return Validated.valid(city);
        }
    }

    public Validated<String, String> validateEmail(String email) {
        if (email == null || email.trim().isEmpty()) {
            return Validated.invalid("Email cannot be empty");
        } else if (!EMAIL_PATTERN.matcher(email).matches()) {
            return Validated.invalid("Invalid email format");
        } else {
            return Validated.valid(email);
        }
    }

    public Validated<String, String> validatePassword(String password) {
        if (password == null || password.trim().isEmpty()) {
            return Validated.invalid("Password cannot be empty");
        } else if (!PASSWORD_PATTERN.matcher(password).matches()) {
            return Validated.invalid(
                "Password must be at least 8 characters long and contain at least one number and one special character");
        } else {
            return Validated.valid(password);
        }
    }

    public Validated<List<String>, String> validatePerson(String name, Integer age, String city,
        String email, String password) {
        List<Validated<String, ?>> validations = List.of(
            validateName(name),
            validateAge(age),
            validateCity(city),
            validateEmail(email),
            validatePassword(password)
        );

        return Validated.mapN(
            values -> "Person validated successfully",
            validations
        );
    }
}

Implementing the Person Controller

Finally, we will implement a REST controller that will expose an endpoint to validate the person’s information.

package com.soumakis.example;

import com.soumakis.example.domain.PersonRequest;
import com.soumakis.example.domain.PersonValidator;
import java.util.List;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/api/person")
public class PersonController {

    private final PersonValidator personValidator;

    public PersonController(PersonValidator personValidator) {
        this.personValidator = personValidator;
    }

    @PostMapping
    public ResponseEntity<?> addPerson(@RequestBody PersonRequest personRequest) {
        Validated<List<String>, String> validationResult = personValidator.validatePerson(
            personRequest.name(),
            personRequest.age(),
            personRequest.city(),
            personRequest.email(),
            personRequest.password()
        );

        if (validationResult.isValid()) {
            return ResponseEntity.noContent().build();
        } else {
            return ResponseEntity.badRequest().body(validationResult.getInvalid());
        }
    }
}

Running the Application

You can run the application by executing the following command:

mvn spring-boot:run

An endpoint is going to be exposed at port 8080. You can send a POST request to http://localhost:8080/api/person with various payloads to test the validation.

{
  "name": "John Doe",
  "age": 25,
  "city": "New York",
  "email": "",
  "password": "Something@123"
}

should fail with a 400 response containing the error message “Email cannot be empty”.

How it differs from a traditional imperative style of using jakarta.validation annotations

The traditional way of validating a form in a Spring Boot application is to use the jakarta.validation annotations. While this approach seems simple and easy to use, it has some limitations. For example, it does not allow you to accumulate errors in a computation. If there are multiple errors in the form, it will short-circuit on the first error and return a 400 response with only that error.

Additionally, the jakarta.validation annotations are not composable. If you have multiple validation rules that depend on each other, it can be challenging to express them using the jakarta.validation annotations.

Moreover, the jakarta.validation annotations are not type-safe. If you make a mistake in the validation logic, you will only find out at runtime when the validation fails. Also, jakarta.validation annotations are not easily testable. You have to run the entire Spring Boot application to test the validation logic whereas with the Validated monad, you can test the validation logic in isolation.

Also the jakarta.validation annotations are using runtime reflection to validate the form which can be slow and error-prone. The Validated monad, on the other hand, is a compile-time construct that is type-safe and efficient.

Finally, it can be used with any framework, not just Spring.

Wait, we can use BindingResult with Spring to accumulate errors in a computation. Why not use that?

Spring provides a way to accumulate errors in a computation using the BindingResult class. You can use the BindingResult class to accumulate errors in a form validation. However, the BindingResult class is not composable. If you have multiple validation rules that depend on each other, it can be challenging to express them using the BindingResult class.

Additionally, using the BindingResult implies that you have correctly configured the @Valid annotation on the controller method parameter. If you forget to add the @Valid annotation, the validation will not be performed, and you will not get any errors.

Should we always use Validated monad for validation?

The Validated monad is a powerful tool for accumulating errors in a computation. However, it is not always necessary to use the Validated monad for validation. If you have a simple form with only a few validation rules that do not depend on each other, you can use the jakarta.validation annotations or the BindingResult class to validate the form.

Conclusion

In this article, we implemented a Validated monad in Java and demonstrated how to use it with Spring Boot to validate a form containing some person specific information and accumulate errors. We compared the Validated monad with the traditional imperative style of using jakarta.validation annotations and the BindingResult class for form validation. We also discussed when to use the Validated monad for validation and when not to use it.

You can find the complete implementation at GitHub

Implementing a custom Spring Transaction Interceptor

Thu, 08 Aug 2024 00:00:00 +0300

In this article, we are going to implement a custom Spring Transaction Interceptor that will allow us to intercept the transactional behavior of our Spring Boot application. We are going to see how we can use this interceptor to allow using the Try monad to handle exceptions in a more functional way while retaining the transactional behavior of the @Transactional annotation we are used to.

For a convenient zero-dependencies library allowing you to start using useful functional programming constructs you can check out the FunctionalUtils library

For an introduction to functional programming you can read here.

Introduction

Spring Boot provides a convenient way to manage transactions using the @Transactional annotation. This annotation allows us to define transactional boundaries around methods and classes, ensuring that the transaction is committed or rolled back based on the method’s outcome.

Traditionally, the preferred way is to throw an exception to signal a failure and rollback the transaction. However, there is another approach for those that wish to embrace functional programming principles. This approach involves using the Try monad to handle exceptions in a more functional way while retaining the transactional behavior of the @Transactional annotation.

Using the Try monad allows us to handle exceptions in a more functional way, providing a more expressive and composable way to deal with errors. The Try monad is a container that represents either a successful computation or a failed computation. It is similar to the Optional monad, but instead of representing the absence of a value, it represents the presence of a value or an exception.

The challenge is to combine the transactional behavior of the @Transactional annotation with the functional error handling provided by the Try monad. In this article, we are going to implement a custom Spring Transaction Interceptor that will allow us to intercept the transactional behavior of our Spring Boot application.

Implementing the custom Spring Transaction Interceptor

To implement a custom Spring Transaction Interceptor, we need to create a class that extends the TransactionInterceptor class provided by Spring.

The TransactionInterceptor class is part of the Spring framework and is used to manage transactions around method invocations. It is responsible for starting, committing, and rolling back transactions based on the method’s outcome.

Our custom interceptor will extend the TransactionInterceptor class and override the invokeWithinTransaction method.

Let’s see the implementation of the custom Spring Transaction Interceptor:

import com.soumakis.control.Try;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.Serial;
import java.lang.reflect.Method;
import java.util.concurrent.atomic.AtomicReference;
import java.util.function.Consumer;
import lombok.SneakyThrows;
import org.aopalliance.intercept.MethodInvocation;
import org.springframework.aop.support.AopUtils;
import org.springframework.beans.factory.BeanFactory;
import org.springframework.lang.Nullable;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.TransactionManager;
import org.springframework.transaction.TransactionStatus;
import org.springframework.transaction.TransactionSystemException;
import org.springframework.transaction.interceptor.DefaultTransactionAttribute;
import org.springframework.transaction.interceptor.TransactionAttribute;
import org.springframework.transaction.interceptor.TransactionAttributeSource;
import org.springframework.transaction.interceptor.TransactionInterceptor;
import org.springframework.transaction.support.CallbackPreferringPlatformTransactionManager;
import org.springframework.util.ClassUtils;

/**
 * CustomTransactionInterceptor is a Spring AOP MethodInterceptor for managing transactions in
 * methods that return Try monad types. It extends TransactionInterceptor to utilize its transaction
 * management functionalities.
 */
public class CustomTransactionInterceptor extends TransactionInterceptor {

  public CustomTransactionInterceptor(TransactionManager transactionManager,
      TransactionAttributeSource tas) {
    super(transactionManager, tas);
  }

  @Override
  @Nullable
  public Object invoke(MethodInvocation invocation) {
    Class<?> targetClass = (invocation.getThis() != null ? AopUtils.getTargetClass(
        invocation.getThis()) : null);

    return invokeWithinTransaction(invocation.getMethod(), targetClass, invocation::proceed);
  }

  /**
   * Invokes the method within a transaction and manages the transaction based on the return type of
   * the method. If the method returns a Try type, it will manage the transaction accordingly.
   * Otherwise, it will manage the transaction as usual.
   *
   * @param method      the Method being invoked
   * @param targetClass the target class that we're invoking the method on
   * @param invocation  the callback to use for proceeding with the target invocation
   * @return the result of the method invocation
   */
  @Override
  @Nullable
  protected Object invokeWithinTransaction(Method method, @Nullable Class<?> targetClass,
      final InvocationCallback invocation) {
    // Retrieves the source of the transaction attribute which can be via spring configuration,
    // programmatic transaction management or annotation based
    // or null if no transaction attribute is found.
    TransactionAttributeSource transactionAttributeSource = getTransactionAttributeSource();
    final TransactionAttribute transactionAttribute = (transactionAttributeSource != null)
        ? transactionAttributeSource.getTransactionAttribute(method, targetClass) : null;
    // Retrieves the transaction manager to be used for managing the transaction
    final TransactionManager transactionManager = determineTransactionManager(transactionAttribute);

    // Typically we operate only on PlatformTransactionManager
    PlatformTransactionManager platformTransactionManager = asPlatformTransactionManager(
        transactionManager);

    // Retrieves the method aop joinpoint identification
    final String joinpointIdentification = methodIdentification(method, targetClass,
        transactionAttribute);

    if (transactionAttribute == null
        || !(platformTransactionManager instanceof CallbackPreferringPlatformTransactionManager)) {
      return handleStandardTransaction(method, platformTransactionManager, transactionAttribute,
          joinpointIdentification, invocation);
    } else {
      return handleCallbackPreferringTransaction(
          method,
          (CallbackPreferringPlatformTransactionManager) platformTransactionManager,
          transactionAttribute, joinpointIdentification, invocation);
    }
  }

  /**
   * Handles a standard transaction by creating a transaction if necessary, proceeding with the
   * method invocation, evaluating the transaction, and cleaning up the transaction info.
   *
   * @param method                     the method being invoked
   * @param platformTransactionManager the transaction manager
   * @param transactionAttribute       the transaction attribute
   * @param joinpointIdentification    the identification of the joinpoint
   * @param invocation                 the callback to use for proceeding with the target
   *                                   invocation
   * @return the result of the method invocation
   */
  private Object handleStandardTransaction(
      Method method,
      PlatformTransactionManager platformTransactionManager,
      TransactionAttribute transactionAttribute,
      String joinpointIdentification, InvocationCallback invocation) {
    TransactionInfo txInfo = createTransactionIfNecessary(platformTransactionManager,
        transactionAttribute, joinpointIdentification);
    AtomicReference<Object> retVal = new AtomicReference<>();

    try {
      retVal.set(invocation.proceedWithInvocation());
      return processTransactionResult(transactionAttribute, txInfo, retVal);
    } catch (Throwable ex) {
      return handleTransactionException(method, ex, txInfo);
    } finally {
      cleanupTransactionInfo(txInfo);
    }
  }

  private Try<Object> handleTransactionException(Method method, Throwable ex,
      TransactionInfo txInfo) {
    rollback(txInfo,
        ignored -> super.completeTransactionAfterThrowing(txInfo, ex));
    if (method.getReturnType().isAssignableFrom(Try.class)) {
      return Try.failure(ex);
    }
    throw new RuntimeException(ex);
  }

  private Object processTransactionResult(TransactionAttribute transactionAttribute,
      TransactionInfo txInfo,
      AtomicReference<Object> retVal) {
    if (transactionAttribute != null) {
      return evaluateTransaction(txInfo, retVal, transactionAttribute);
    }
    // It means that no transaction is demarcated.
    return retVal;
  }

  /**
   * Handles a callback preferring transaction by executing the transaction, proceeding with the
   * method invocation, evaluating the transaction, and cleaning up the transaction info.
   *
   * @param method                     the Method being invoked
   * @param platformTransactionManager the transaction manager
   * @param transactionAttribute       the transaction attribute
   * @param joinpointIdentification    the identification of the joinpoint
   * @param invocation                 the callback to use for proceeding with the target
   *                                   invocation
   * @return the result of the method invocation
   */
  private Object handleCallbackPreferringTransaction(
      Method method,
      CallbackPreferringPlatformTransactionManager platformTransactionManager,
      TransactionAttribute transactionAttribute, String joinpointIdentification,
      InvocationCallback invocation) {
    try {
      return platformTransactionManager.execute(transactionAttribute, status -> {
        TransactionInfo txInfo = prepareTransactionInfo(platformTransactionManager,
            transactionAttribute, joinpointIdentification, status);
        try {
          return invocation.proceedWithInvocation();
        } catch (Throwable ex) {
          return handleTransactionException(method, ex, txInfo);
        } finally {
          cleanupTransactionInfo(txInfo);
        }
      });
    } catch (TransactionSystemException ex) {
      return Try.failure(ex);
    }
  }

  /**
   * Evaluates the transaction by committing the transaction after returning and returning the
   * result of the method invocation.
   *
   * @param txInfo               the transaction info
   * @param retVal               the result of the method invocation
   * @param transactionAttribute the transaction attribute
   * @return the result of the method invocation
   */
  private Object evaluateTransaction(TransactionInfo txInfo,
      AtomicReference<Object> retVal, TransactionAttribute transactionAttribute) {
    TransactionStatus status = txInfo.getTransactionStatus();
    if (status != null && (retVal.get() instanceof Try<?>)) {
      retVal.set(evaluateTryFailure(retVal, transactionAttribute, status));
      try {
        return commitTransaction(txInfo, retVal);
      } catch (Exception e) {
        // For any exception do not propagage the exception but respect the return type and return a Try#Failure.
        return Try.failure(e);
      }
    }
    return commitTransaction(txInfo, retVal);
  }

  private Object commitTransaction(TransactionInfo txInfo, AtomicReference<Object> retVal) {
    commitTransactionAfterReturning(txInfo);
    return retVal.get();
  }

  @Serial
  private void writeObject(ObjectOutputStream oos) throws IOException {
    oos.defaultWriteObject();
    oos.writeObject(getTransactionManagerBeanName());
    oos.writeObject(getTransactionManager());
    oos.writeObject(getTransactionAttributeSource());
    oos.writeObject(getBeanFactory());
  }

  @Serial
  private void readObject(ObjectInputStream ois) throws IOException, ClassNotFoundException {
    ois.defaultReadObject();
    setTransactionManagerBeanName((String) ois.readObject());
    setTransactionManager((PlatformTransactionManager) ois.readObject());
    setTransactionAttributeSource((TransactionAttributeSource) ois.readObject());
    setBeanFactory((BeanFactory) ois.readObject());
  }

  private PlatformTransactionManager asPlatformTransactionManager(
      @Nullable Object transactionManager) {
    if (transactionManager == null) {
      return null;
    }
    if (transactionManager instanceof PlatformTransactionManager ptm) {
      return ptm;
    } else {
      throw new IllegalStateException(
          "Specified transaction manager is not a PlatformTransactionManager: "
              + transactionManager);
    }
  }

  private String methodIdentification(Method method, @Nullable Class<?> targetClass,
      @Nullable TransactionAttribute transactionAttribute) {
    String methodIdentification = methodIdentification(method, targetClass);
    if (methodIdentification == null) {
      if (transactionAttribute instanceof DefaultTransactionAttribute dta) {
        methodIdentification = dta.getDescriptor();
      }
      if (methodIdentification == null) {
        methodIdentification = ClassUtils.getQualifiedMethodName(method, targetClass);
      }
    }
    return methodIdentification;
  }

  @SneakyThrows
  private void rollback(TransactionInfo txInfo, Consumer<?> rollbackAction) {
    if (txInfo.getTransactionStatus() != null) {
      rollbackAction.accept(null);
    }
  }

  private static Object evaluateTryFailure(AtomicReference<Object> retVal,
      TransactionAttribute txAttr,
      TransactionStatus status) {
    return ((Try<?>) retVal.get()).onFailure(ex -> {
      // This basically will respect the @Transactional(noRollbackFor= {...})
      if (txAttr.rollbackOn(ex)) {
        status.setRollbackOnly();
      }
    });
  }
}

In the code above, we have implemented a custom Spring Transaction Interceptor that extends the TransactionInterceptor class provided by Spring. The custom interceptor overrides the “invokeWithinTransaction” method to manage transactions around method invocations.

Specifically, the important points are:

When committing the transaction, we evaluate the result of the method invocation. If the method returns a Try type, we evaluate the Try failure and set the transaction status to roll back if necessary. Otherwise, we commit the transaction.
When an exception occurs, we roll back the transaction and return a Try failure if the method returns a Try type. Otherwise, we throw a RuntimeException containing the thrown exception.
We handle both standard transactions and callback preferring transactions. Callback preferring transactions are transactions that prefer a callback-based approach to executing the transaction.
We have implemented the writeObject and readObject methods to serialize and deserialize the custom interceptor.

Example project

You can find the complete implementation of the custom Spring Transaction Interceptor along with an example on how to use it along with integration tests in the following GitHub repository spring-custom-transaction-interceptor.

Conclusion

In this article, we have implemented a custom Spring Transaction Interceptor that allows us to intercept the transactional behavior of our Spring Boot application. We have seen how we can use this interceptor to allow using the Try monad to handle exceptions in a more functional way while retaining the transactional behavior of the @Transactional annotation.

By learning how to implement a custom Spring Transaction Interceptor, you can combine the transactional behavior of Spring with your specific use case requirements.

Happy coding and remember to subscribe to the newsletter to get the latest updates every week!

How the JDK Optional breaks the monad laws and why it matters

Thu, 01 Aug 2024 00:00:00 +0300

In this article, we are going to prove why the JDK Optional fails to respect the Monad Laws and why it matters.

For a convenient zero-dependencies library allowing you to start using useful functional programming constructs you can check out the FunctionalUtils library

Introduction

For an introduction to functional programming you can read here.

What are monad laws?

Monad laws dictate the behavior of a monad in the functional programming world and whether it can be considered a true one or not. There are three laws, namely:

Left Identity
Right Identity
Associativity

Let’s analyze each law:

Left identity: The left identity states that given a function ‘f’ returning a monad then flatMap(unit(value), f) === f(value). In other words, considering the Optional from the JDK we can state the left identity as ‘of(value).flatMap(f) === f(value)’.
Right identity: The right identity states the given a function ‘f’ returning a monad and ‘m’ being an instance of a monad then flatMap(m, unit) === m. In other words, considering the Optional from the JDK we can state the right identity as ‘m.flatMap(Optional::of) === m’.
Associativity: If we have a chain of monadic function applications, it does not matter how they are nested. It can be represented as flatMap(flatMap(m, f), g) === flatMap(m, x -> g(f(x))) where ‘m’ is an instance of a monad, ‘f’ a function that returns a monad and ‘g’ another function as well. In other words, considering again the Optional from the JDK we can state the associativity as: ‘m.flatMap(f).flatMap(g) === m.flatMap(x -> f.apply(x).flatMap(g))’

Why they matter

Monad laws ensure that the composition of monadic functions behaves predictably. This predictability simplifies reasoning about code and makes refactoring safer.
By adhering to monad laws, developers can create more maintainable code. Functions and monadic chains can be rearranged and refactored without unexpected side effects, leading to clearer and more modular code.

How JDK Optional breaks the monad laws

Let’s see an example for each monad law and try to figure out if Optional respects it.

Left identity

Consider a function ‘f’ that given an Integer returns an Optional based on some business logic.

Function<Integer, Optional<Integer>> f = x -> {
    if (x == null) {
        x = -1;
    } else if (x == 2) {
        x = null;
    } else {
        x = x + 1;
    }
    return Optional.ofNullable(x);
};

Optional.of(1).flatMap(f).equals(f.apply(1)) // This is true which means Optional[2] === Optional[2]
Optional.of(2).flatMap(f).equals(f.apply(2)); // This is also true which means Optional.empty === Optional.empty
Optional.ofNullable((Integer) null).flatMap(f).equals(f.apply(null)); // This is false because the left hand side results in Optional.empty and the right side results in Optional[1] breaking the left identity law.

Right identity

Optional<Integer> opt = Optional.of(1);

opt.flatMap(Optional::of); // This should be equivalent to opt

Optional<Integer> opt2 = Optional.empty();

opt2.flatMap(Optional::of); // This should be equivalent to opt2 which is empty.

The right identity law holds as we can see.

Associativity

Let’s consider two basic functions:

Function<Integer, Option<Integer>> f = x -> Option.of(x + 1);
Function<Integer, Option<Integer>> g = x -> Option.of(x * 2);

And the application of these two functions

Optional<Integer> opt = Optional.of(1);
Optional<Integer> result1 = opt.flatMap(f).flatMap(g);
Optional<Integer> result2 = opt.flatMap(x -> f.apply(x).flatMap(g));

Left Side: opt.flatMap(f).flatMap(g)

opt.flatMap(f):
- opt is Optional.of(1).
- Applying f to 1 results in Optional.of(2).
Optional.of(2).flatMap(g):
- Applying g to 2 results in Optional.of(4).

So, result1 is Optional.of(4).

Right Side: opt.flatMap(x -> f.apply(x).flatMap(g))

opt.flatMap(x -> f.apply(x).flatMap(g)):
- opt is Optional.of(1).
Applying x -> f.apply(x).flatMap(g) to 1:
- f.apply(1) results in Optional.of(2).
- Applying g to 2 results in Optional.of(4).

So results1 == result2 thus **Associativity ** is respected.

However, sadly this is not always the case. Let’s explore another more real-world scenario.

Example scenario where associativity is broken

Fetching user data from a database.
Process the data
Format the data

record User(int id, String name) {}

static Function<Integer, Optional<User>> fetchUser = id -> {
    if (id == 1) {
        return Optional.of(new User(1, "John"));
    } else {
        return Optional.empty();
    }
};

static Function<User, Integer> fetchBalance = user -> user.id() == 1 ? null : 100;

static Function<Integer, String> formatBalance = balance -> {
    if (balance == null) {
        return "No Balance";
    } else {
        return "Balance: " + balance;
    }
};

We have defined three functions that represent the three steps of our scenario.

fetchUser: Given an id it returns the user data.
fetchBalance: Given the user data, it returns the balance.
formatBalance: Given the balance, it returns a formatted string.

Although the example might seem trivial, it is easy to imagine that the fetchBalance and formatBalance could be more complex involving a lot of more business logic and potentially being outside our control such as in libraries and potentially being a legacy code.

Let’s see how the associativity law is broken in this case.

The original code we have in our code and we want to refactor is the following:

String result = fetchUser.apply(1)
      .map(fetchBalance.andThen(formatBalance))
      .orElse("Processing could not be completed");

System.out.println(result);

fetchUser.apply(1).map(fetchBalance.andThen(formatBalance)) is a composition of two functions inside a map function.

fetchUser.apply(1) returns Optional[User(1, “John”)]
fetchBalance is applied to the User(1, “John”) and returns null
formatBalance is applied to null passed from fetchBalance and returns “No Balance”
The message displayed is “No Balance”

We are tasked of refactoring the above code to be composed of a series of monadic function applications like:

String result = fetchUser.apply(1)
      .map(fetchBalance)
      .map(formatBalance)
      .orElse("Processing could not be completed");

System.out.println(result);

fetchUser.apply(1).map(fetchBalance).map(formatBalance) is a series of monadic function applications.

fetchUser.apply(1) returns Optional[User(1, “John”)]
map(fetchBalance) returns Optional[null] which is translated to Optional.empty
No further processing is done as the Optional is empty and the message displayed is “Processing could not be completed”.

Sadly, the associativity law is broken in this case. The refactored code does not behave the same as the original code thus introducing subtle bugs and making the code harder to debug. If we don’t have enough test coverage, we might not even notice the bug until the runtime.

What we can do about it

There are few options when it comes to the JDK Optional. We can either:

Use a different library: There are many libraries out there that provide a more robust implementation of the Optional monad. For example, the Vavr library provides an Option type that respects the monad laws. However, it is not maintained anymore. Another option is to use the FunctionalUtils library which provides a more robust and tested implementation of the Optional monad.
Implement your own Optional: Take a look at how to implement Your own Option monad upholding all laws. It is straightforward to implement your own Option monad that respects the monad laws and additionally maintain full compatibility with the JDK Optional.
Use the Optional with caution: If you are aware of the limitations of the JDK Optional and the potential issues it can cause, you can still use it but with caution. Make sure to thoroughly test your code and be aware of the potential pitfalls.

References

https://www.sitepoint.com/how-optional-breaks-the-monad-laws-and-why-it-matters/

Immutable Java ArrayList Monad

Sat, 27 Jul 2024 00:00:00 +0300

In this article, we are going to demonstrate how to implement an immutable ArrayList-Like Monad in Java. We will discuss the pros and cons of this approach and how it can be used to enhance a more functional programming style in Java.

For an introduction to functional programming you can read here

For implementing a persistent and immutable LinkedList you can read here

For the complete implementation and many other useful monads and structures you can check the FunctionalUtils library

Introduction

What is an ArrayList

An ArrayList is a resizable array implementation in Java. It is a part of the Java Collections Framework and is implemented in the java.util package. It is a dynamic array that can grow or shrink in size. It is similar to an array but has more functionalities than a simple array.

An ArrayList is mutable in nature, which means that the elements of an ArrayList can be changed or modified. This has the implication that extra care needs to be taken when sharing the ArrayList between different parts of the code or when accessing the list from multiple threads as it is not a thread safe data structure.

What exists for ensuring thread safety for an ArrayList

Java provides a Collections.synchronizedList method that wraps an ArrayList and returns a synchronized list.
Java provides a CopyOnWriteArrayList class that is a thread-safe and mutable variant of ArrayList in which all mutative operations are implemented by making a fresh copy of the underlying array.

What is an Immutable ArrayList

An immutable ArrayList is a list that cannot be changed after it is created. This means that once the list is created, it cannot be modified. Any operation that would modify the list will instead create a new list with the modified elements. This has the implication that the list is thread safe and can be shared between different parts of the code without any issues.

Performance wise, an immutable list is not as efficient as a mutable list as it requires creating a new list every time a modification is made. However, the benefits of immutability are that it is thread safe and can be shared between different parts of the code without any issues.

Proposal

Our immutable variation of the known Java ArrayList will be called ArraySeq. It will be a monad that will provide functional programming capabilities to Java. The ArraySeq will be an immutable list that will provide the following functional programming capabilities:

Map: Apply a function to each element of the list and return a new list with the results.
Filter: Filter the elements of the list based on a predicate and return a new list with the filtered elements.
FlatMap: Apply a function to each element of the list and return a new list with the results. The function returns a list and the results are concatenated.
Update: Update an element at a specific index and return a new list with the updated element.
Append: Append an element to the end of the list and return a new list with the appended element.
Prepend: Prepend an element to the start of the list and return a new list with the prepended element.
Delete: Delete an element at a specific index and return a new list without the deleted element.
Retrieve: Retrieve an element at a specific index.

Differences between ArraySeq and Collections.unmodifiableList

Collections.unmodifiableList returns a view of the list that is unmodifiable. This means that the original list can still be modified and the unmodifiable list will reflect these changes. In contrast, the ArraySeq is an immutable list that cannot be modified after it is created.
Collections.unmodifiableList is not a monad and does not provide functional programming capabilities such as map, filter, flatMap, etc. In contrast, the ArraySeq is a monad that provides these functional programming capabilities.
Collections.unmodifiableList is not thread safe. In contrast, the ArraySeq is an immutable list that is thread safe and can be shared between different parts of the code without any issues.
Collections.unmodifiableList will throw an UnsupportedOperationException if a modification operation is called. In contrast, the ArraySeq will return a new list with the modified elements. This means that the ArraySeq is more predictable as the frequent problem with Collections.unmodifiableList is that typically a java.util.List is returned and cannot be distinguished from a mutable list.

What is the subtle issue with Java 9+ List.of method and Guava ImmutableList

When working with Java 9+ and using the List.of method to create an immutable list or using Guava’s ImmutableList, it is important to note that:

In the first case the List.of method returns a java.util.List which is not distinguishable in the application code from any other typically mutable implementation of the List interface. This may result in accidental UnsupportedOperationException being thrown at runtime.
Guava’s ImmutableList still is a subtype of java.util.List and has the same issues as the List.of method.

This is a design issue with both Guava and JDK which they are treating immutable values wrongly and resulting in exceptions thrown at runtime which can be avoided by using a proper immutable list implementation.

Implementation

public final class ArraySeq<T> {

  private final T[] elements;
  private final int length;

  private ArraySeq(T[] elements) {
    this.elements = elements;
    length = elements.length;
  }

  /**
   * Creates a new sequence with the given elements.
   *
   * @param elements the elements to add
   * @param <T>      the type of the elements
   * @return a new sequence with the elements added
   */
  @SafeVarargs
  public static <T> ArraySeq<T> of(T... elements) {
    return new ArraySeq<>(elements);
  }

  /**
   * Creates an empty sequence.
   *
   * @param <T> the type of the elements
   * @return an empty sequence
   */
  @SuppressWarnings("unchecked")
  public static <T> ArraySeq<T> empty() {
    return new ArraySeq<>((T[]) new Object[0]);
  }

  /**
   * Returns the length of the sequence.
   *
   * @return the length of the sequence
   */
  public int length() {
    return length;
  }

  /**
   * Appends the given elements to the sequence. Since it is an immutable data structure, it returns
   * a new sequence with the elements appended.
   *
   * @param elementsToAppend the elements to append
   * @return a new sequence with the elements appended
   */
  @SuppressWarnings("unchecked")
  public ArraySeq<T> append(T... elementsToAppend) {
    var newElements = (T[]) new Object[length + elementsToAppend.length];
    System.arraycopy(this.elements, 0, newElements, 0, length);
    System.arraycopy(elementsToAppend, 0, newElements, length, elementsToAppend.length);
    return new ArraySeq<>(newElements);
  }

  /**
   * Appends the given sequence to the sequence. Since it is an immutable data structure, it returns
   * a new sequence with the elements appended.
   *
   * @param other the sequence to append
   * @return a new sequence with the elements appended
   */
  public ArraySeq<T> append(ArraySeq<T> other) {
    return append(other.elements);
  }

  /**
   * Prepends the given elements to the sequence. Since it is an immutable data structure, it
   * returns a new sequence with the elements prepended.
   *
   * @param elementsToPrepend the elements to prepend
   * @return a new sequence with the elements prepended
   */
  @SuppressWarnings("unchecked")
  public ArraySeq<T> prepend(T... elementsToPrepend) {
    var newElements = (T[]) new Object[length + elementsToPrepend.length];
    System.arraycopy(elementsToPrepend, 0, newElements, 0, elementsToPrepend.length);
    System.arraycopy(this.elements, 0, newElements, elementsToPrepend.length, length);
    return new ArraySeq<>(newElements);
  }

  /**
   * Updates the element at the given index with the updated value. Since it is an immutable data
   * structure, it returns a new sequence with the element updated.
   *
   * @param index        the index of the element to update
   * @param updatedValue the updated value
   * @return a new sequence with the element updated
   * @throws IndexOutOfBoundsException if the index is out of bounds
   */
  @SuppressWarnings("unchecked")
  public ArraySeq<T> update(int index, T updatedValue) {
    checkBounds(index);
    var newElements = (T[]) new Object[length];
    System.arraycopy(elements, 0, newElements, 0, length);
    newElements[index] = updatedValue;
    return new ArraySeq<>(newElements);
  }

  /**
   * Updates the element at the given index with the evaluation of the computation represented as an
   * {@link  Supplier}. Since it is an immutable data structure, it returns a new sequence with the
   * element updated.
   *
   * @param index                      the index of the element to update
   * @param computationForUpdatedValue the computation to evaluate for the updated value
   * @return a new sequence with the element updated
   * @throws IndexOutOfBoundsException if the index is out of bounds
   */
  @SuppressWarnings("unchecked")
  public ArraySeq<T> update(int index, Supplier<T> computationForUpdatedValue) {
    checkBounds(index);
    var newElements = (T[]) new Object[length];
    System.arraycopy(elements, 0, newElements, 0, length);
    newElements[index] = computationForUpdatedValue.get();
    return new ArraySeq<>(newElements);
  }

  /**
   * Returns the element at the given index.
   *
   * @param index the index of the element to get
   * @return the element at the given index
   * @throws IndexOutOfBoundsException if the index is out of bounds
   */
  public T get(int index) {
    checkBounds(index);
    return elements[index];
  }

  /**
   * Deletes the element at the given index. Since it is an immutable data structure, it returns a
   * new sequence with the element deleted.
   *
   * @param index the index of the element to delete
   * @return a new sequence with the element deleted
   * @throws IndexOutOfBoundsException if the index is out of bounds
   */
  @SuppressWarnings("unchecked")
  public ArraySeq<T> delete(int index) {
    checkBounds(index);
    var newElements = (T[]) new Object[length - 1];
    System.arraycopy(elements, 0, newElements, 0, index);
    System.arraycopy(elements, index + 1, newElements, index, length - index - 1);
    return new ArraySeq<>(newElements);
  }

  /**
   * Maps the elements of the sequence to a new sequence using the given function. The complexity of
   * this method is O(n) where n is the number of elements.
   *
   * @param fn  the function to map the elements
   * @param <U> the type of the elements of the new sequence
   * @return a new sequence with the elements mapped
   */
  @SuppressWarnings("unchecked")
  public <U> ArraySeq<U> map(Function<? super T, ? extends U> fn) {
    var newElements = (U[]) new Object[length];
    for (int i = 0; i < length; i++) {
      newElements[i] = fn.apply(elements[i]);
    }
    return new ArraySeq<>(newElements);
  }

  /**
   * Applies a flat map function to the elements of the sequence. The complexity of this method is
   * O(n) where n is the number of elements.
   *
   * @param fn  the flat map function
   * @param <U> the type of the elements of the new sequence
   * @return a new sequence with the elements flat mapped
   */
  public <U> ArraySeq<U> flatMap(Function<? super T, ? extends ArraySeq<U>> fn) {
    List<U> temp = new ArrayList<>();

    for (int i = 0; i < length; i++) {
      ArraySeq<U> mapped = fn.apply(elements[i]);
      for (int j = 0; j < mapped.length; j++) {
        temp.add(mapped.get(j));
      }
    }

    @SuppressWarnings("unchecked")
    U[] finalArray = temp.toArray((U[]) new Object[0]);

    return new ArraySeq<>(finalArray);
  }

  /**
   * Filters the elements of the sequence using the given predicate. The complexity of this method
   * is O(n) where n is the number of elements.
   *
   * @param predicate the predicate to filter the elements
   * @return a new sequence with the elements filtered
   */
  public ArraySeq<T> filter(Predicate<T> predicate) {
    List<T> temp = new ArrayList<>();

    for (int i = 0; i < length; i++) {
      var element = elements[i];
      if (predicate.test(element)) {
        temp.add(element);
      }
    }

    @SuppressWarnings("unchecked")
    T[] finalArray = temp.toArray((T[]) new Object[0]);

    return new ArraySeq<>(finalArray);
  }

  /**
   * Converts the sequence to a {@link List}.
   * <p>
   * <b>NOTE: This method might break the immutability if the underlying elements of the generated
   * ArrayList are mutable</b>
   *
   * @return a {@link List} with the elements of the sequence
   */
  public List<T> toJavaArrayList() {
    return Arrays.asList(Arrays.copyOf(elements, length));
  }

  /**
   * Converts the sequence to a {@link Stream}.
   *
   * <p>
   * <b>NOTE: This method might break the immutability if the underlying elements of the generated
   * Stream are mutable</b>
   * </p>
   *
   * @return a {@link Stream} with the elements of the sequence
   */
  public Stream<T> stream() {
    return Arrays.stream(elements);
  }

  @Override
  public boolean equals(Object o) {
    if (this == o) {
      return true;
    }
    if (o == null || getClass() != o.getClass()) {
      return false;
    }
    ArraySeq<?> arraySeq = (ArraySeq<?>) o;
    return length == arraySeq.length && Objects.deepEquals(elements, arraySeq.elements);
  }

  @Override
  public int hashCode() {
    return Objects.hash(Arrays.hashCode(elements), length);
  }

  private void checkBounds(int index) {
    if (index < 0 || index >= length) {
      throw new IndexOutOfBoundsException("Index out of bounds: " + index);
    }
  }

}

Notes:

The ArraySeq class is a final class to prevent inheritance.
The ArraySeq class is generic to allow the user to define the type of the elements.
The ArraySeq does not perform any deep copy of the elements. This means that if the elements are mutable, the immutability of the ArraySeq will be broken.
The ArraySeq class is an immutable data structure. This means that all the methods that would modify the list return a new list with the modified elements. This also means that although System.arraycopy is used to copy the elements, since it does not use any advanced data structure such as a trie, the complexity of the operations is O(n) where n is the number of elements. It is also not as efficient as a mutable list for large lists, but it is a great choice when immutability, thread safety and functional programming composition are more important than performance.
The ArraySeq class is a true monad as it provides the map, flatMap, of methods and also respect all monad laws.

Usage

public class ArraySeqExample {

  public static void main(String[] args) {
    ArraySeq<Integer> arraySeq = ArraySeq.of(1, 2, 3, 4, 5);

    // Map
    ArraySeq<Integer> mapped = arraySeq.map(x -> x * 2);
    System.out.println(mapped.toJavaArrayList()); // [2, 4, 6, 8, 10]

    // Filter
    ArraySeq<Integer> filtered = arraySeq.filter(x -> x % 2 == 0);
    System.out.println(filtered.toJavaArrayList()); // [2, 4]

    // FlatMap
    ArraySeq<Integer> flatMapped = arraySeq.flatMap(x -> ArraySeq.of(x, x * 2));
    System.out.println(flatMapped.toJavaArrayList()); // [1, 2, 2, 4, 3, 6, 4, 8, 5, 10]

    // Append
    ArraySeq<Integer> appended = arraySeq.append(6, 7, 8);
    System.out.println(appended.toJavaArrayList()); // [1, 2, 3, 4, 5, 6, 7, 8]

    // Prepend
    ArraySeq<Integer> prepended = arraySeq.prepend(0, -1, -2);
    System.out.println(prepended.toJavaArrayList()); // [0, -1, -2, 1, 2, 3, 4, 5]

    // Update
    ArraySeq<Integer> updated = arraySeq.update(2, 10);
    System.out.println(updated.toJavaArrayList()); // [1, 2, 10, 4, 5]

    // Delete
    ArraySeq<Integer> deleted = arraySeq.delete(2);
    System.out.println(deleted.toJavaArrayList()); // [1, 2, 4, 5]

    // Retrieve
    System.out.println(arraySeq.get(2)); // 3
  }

}

Conclusion

In this article, we have demonstrated how to implement an immutable ArrayList-Like Monad in Java. We have discussed the pros and cons of this approach and how it can be used to implement a functional programming style in Java. The ArraySeq class is a true monad as it provides the map, flatMap, of methods and also respect all monad laws. It is a great choice when immutability, thread safety and functional programming composition are more important than performance.

Implementing an Option Monad in Java

Wed, 24 Jul 2024 00:00:00 +0300

In this article we are going to explain what is the Option monad, its purpose, why it is a real enhancement over the JDK Optional, and how to implement it from scratch.

For an introduction to functional programming you can read here

For other very useful monads and functional programming utilities you can checkout this library of mine here

Introduction

What is an Option monad

An Option monad is a type that represents the presence or absence of a value. It can be used to avoid null pointer exceptions and to make the code more readable and maintainable. It also enjoys the benefits of immutability and chaining computations inherent in monads.

Why is it a real enhancement over the JDK Optional

It is a real enhancement over the JDK Optional class because it provides the following benefits:

Avoids NoSuchElementException: The Option monad defines an ADT with two cases: Some and None. This makes it impossible to access a value that is not present because there is not get method in the Option interface at all.
Avoid NullPointerException: The Option monad handles the null by instantiating an instance of None.
Support for Pattern Matching: Since the Option monad is an ADT, it can be pattern matched. This allows for more expressive and readable code.

Whereas the JDK Optional class lacks these features as it erroneously results in:

Optional.of throws NullPointerException if the provided value is null. This defeats the purpose of having an Optional to handle potentially null values gracefully. It forces the developer to handle nulls before calling of or worse using ofNullable everywhere.
get on an empty Optional throws NoSuchElementException if the Optional is empty. This encourages a pattern where the developer must always check if the Optional is present before calling get, leading to something that is not safer than checking for nulls directly.

Implementation

Before proceeding with the implementation, we need to outline our goals. Our Option monad should achieve the following:

Immutability: Every operation should be immutable.
Transformation and Chaining: Provide map and flatMap methods to transform and chain computations in a functional programming style.
Pattern Matching: Model the Option as an ADT with two cases: Some and None.
Avoid NullPointerException: Handle the null case by instantiating an instance of None.
Avoid NoSuchElementException: Do not provide a get method in the Option interface.
Thread Safety: Every operation in the Option monad should be thread-safe.

Let’s first define the Some and None classes:

public record Some<T>(T value) implements Option<T> {

}

public record None<T>() implements Option<T> {

}

The Option interface is defined as follows:

public sealed interface Option<T> permits Some, None {

  /**
   * Returns an {@code Option} instance that wraps the given value. If the value is {@code null}
   * then an instance of {@link None} is returned.
   *
   * @param value the value to wrap
   * @param <T>   the type of the value
   * @return an {@code Option} instance
   */
  static <T> Option<T> of(T value) {
    return value == null ? new None<>() : new Some<>(value);
  }

  /**
   * Returns a {@code None} instance.
   *
   * @param <T> the type of the value
   * @return a {@code None} instance
   */
  static <T> Option<T> none() {
    return new None<>();
  }

  /**
   * Returns the value if it exists, otherwise returns the given default value.
   *
   * @param defaultValue the default value to return if the value does not exist
   * @return the value if it exists, otherwise the default value
   */
  default T getOrElse(T defaultValue) {
    return switch (this) {
      case Some<T> some -> some.value();
      case None<T> ignored -> defaultValue;
    };
  }

  /**
   * Returns whether the value exists.
   *
   * @return {@code true} if the value does not exist, otherwise {@code false}
   */
  default boolean isEmpty() {
    return this instanceof None;
  }

  /**
   * Returns whether the value exists.
   *
   * @return {@code true} if the value exists, otherwise {@code false}
   */
  default boolean isDefined() {
    return this instanceof Some;
  }

  /**
   * Applies the given predicate to the value if it the instance is a {@link Some} and returns a the
   * instance of {@link Some} if the predicate is satisfied, otherwise returns an instance of
   * {@link None}.
   *
   * @param predicate the predicate to apply to the value
   * @return an {@code Option} instance
   */
  default Option<T> filter(Predicate<T> predicate) {
    return switch (this) {
      case Some<T> some -> predicate.test(some.value()) ? this : new None<>();
      case None<T> ignored -> this;
    };
  }

  /**
   * Applies the first function if the instance is a {@link None} and the second function if the
   * instance is a {@link Some}.
   *
   * @param notFoundFunction the function to apply if the instance is a {@link None}
   * @param foundFunction    the function to apply if the instance is a {@link Some}
   * @param <U>              the type of the result
   * @return the result of the applied function
   */
  default <U> U fold(Supplier<U> notFoundFunction,
      Function<? super T, ? extends U> foundFunction) {
    Objects.requireNonNull(notFoundFunction);
    Objects.requireNonNull(foundFunction);
    return switch (this) {
      case Some<T> some -> foundFunction.apply(some.value());
      case None<T> ignored -> notFoundFunction.get();
    };
  }

  /**
   * Maps the value to a new value using the given function if the instance is a {@link Some},
   * otherwise returns an instance of {@link None}.
   *
   * @param fn  the function to apply to the value
   * @param <U> the type of the new value
   * @return an {@code Option} instance
   */
  default <U> Option<U> map(Function<? super T, ? extends U> fn) {
    Objects.requireNonNull(fn);
    return switch (this) {
      case Some<T> some -> Option.of(fn.apply(some.value()));
      case None<T> ignored -> Option.none();
    };
  }

  /**
   * Flat Maps the value to a new value using the given function if the instance is a {@link Some},
   * otherwise returns an instance of {@link None}.
   *
   * @param fn  the function to apply to the value
   * @param <U> the type of the new value
   * @return an {@code Option} instance
   */
  default <U> Option<U> flatMap(Function<? super T, ? extends Option<U>> fn) {
    Objects.requireNonNull(fn);
    return switch (this) {
      case Some<T> some -> fn.apply(some.value());
      case None<T> ignored -> Option.none();
    };
  }

  /**
   * Returns an instance of {@link Right} if the instance is a {@link Some}, otherwise returns an
   * instance of {@link Left}.
   *
   * @param supplier the supplier to provide the value for the left side
   * @param <X>      the type of the left side
   * @return an instance of {@link Either}
   */
  default <X> Either<X, T> toRight(Supplier<X> supplier) {
    Objects.requireNonNull(supplier);
    return switch (this) {
      case Some<T> some -> Either.right(some.value());
      case None<T> ignored -> Either.left(supplier.get());
    };
  }

  /**
   * Returns an instance of {@link Left} if the instance is a {@link Some}, otherwise returns an
   * instance of {@link Right}.
   *
   * @param supplier the supplier to provide the value for the right side
   * @param <X>      the type of the right side
   * @return an instance of {@link Either}
   */
  default <X> Either<T, X> toLeft(Supplier<X> supplier) {
    Objects.requireNonNull(supplier);
    return switch (this) {
      case Some<T> some -> Either.left(some.value());
      case None<T> ignored -> Either.right(supplier.get());
    };
  }

  /**
   * Returns a {@link Try} instance that wraps the value if the instance is a {@link Some},
   * otherwise returns a {@link Try} instance that wraps the exception provided by the supplier.
   *
   * @param exceptionSupplier the supplier to provide the exception
   * @return a {@link Try} instance
   */
  default Try<T> toTry(Supplier<Exception> exceptionSupplier) {
    Objects.requireNonNull(exceptionSupplier);
    return switch (this) {
      case Some<T> some -> Try.success(some.value());
      case None<T> ignored -> Try.failure(exceptionSupplier.get());
    };
  }

  /**
   * Returns an {@link Optional} instance that wraps the value if the instance is a {@link Some},
   * otherwise returns an empty {@link Optional}.
   *
   * @return an {@link Optional} instance
   */
  default Optional<T> toJavaOptional() {
    return switch (this) {
      case Some<T> some -> Optional.of(some.value());
      case None<T> ignored -> Optional.empty();
    };
  }

  /**
   * Returns the original Option if it is a {@link Some}, otherwise returns the Option provided by
   * the supplier.
   *
   * @param supplier the supplier to provide the Option
   * @return an {@code Option} instance
   */
  @SuppressWarnings("unchecked")
  default Option<T> or(Supplier<? extends Option<? extends T>> supplier) {
    Objects.requireNonNull(supplier);
    return switch (this) {
      case Some<T> ignored -> this;
      case None<T> ignored -> (Option<T>) supplier.get();
    };
  }

}

Examples

Option<String> some = Option.of("Hello");
Option<String> none = Option.none();

// map
Option<Integer> length = some.map(String::length);

// flatMap
Option<Integer> length2 = some.flatMap(s -> Option.of(s.length()));

// filter
Option<String> filtered = some.filter(s -> s.length() > 5);

// getOrElse
String value = none.getOrElse("World");

// isEmpty
boolean empty = none.isEmpty(); // should be true

// isDefined
boolean defined = some.isDefined(); // should be true

// fold
String result = some.fold(() -> "Not Found", Function.identity());
String result2 = none.fold(() -> "Not Found", Function.identity()); // should be "Not Found"

// toRight
Either<String, String> right = some.toRight(() -> "Something went wrong"); // should be Right("Hello")
Either<String, String> left = none.toRight(() -> "Something went wrong"); // should be Left("Something went wrong")

// toLeft
Either<String, String> left2 = some.toLeft(() -> "Something went wrong"); // should be Left("Hello")
Either<String, String> right2 = none.toLeft(() -> "Something went wrong"); // should be Right("Something went wrong")

// toTry
Try<String> success = some.toTry(() -> new Exception("Something went wrong")); // should be Success("Hello")
Try<String> failure = none.toTry(() -> new Exception("Something went wrong")); // should be Failure(Exception("Something went wrong"))

// toJavaOptional
Optional<String> optional = some.toJavaOptional(); // should be Optional("Hello")
Optional<String> emptyOptional = none.toJavaOptional(); // should be Optional.empty()

// or
Option<String> or = none.or(() -> Option.of("World")); // should be Some("World")

Conclusion

In this article, we have implemented an Option monad from scratch in Java. We have seen how the Option monad can be used to avoid null pointer exceptions and make the code more readable and maintainable. We have also discussed the benefits of the Option monad over the JDK Optional class and outlined the goals of our implementation. Finally, we have provided examples of how to use the Option monad in practice.

Implementing a Lazy Monad in Java

Tue, 23 Jul 2024 00:00:00 +0300

In this article we are going to explain what is a Lazy monad, its purpose and use cases and implement it from scratch.

For an introduction to functional programming you can read here

For other very useful monads and functional programming utilities you can checkout this library of mine here

Introduction

What is a Lazy monad

A lazy monad allows us to defer the execution of computations. With this monad, we enjoy all the benefits of chaining computations and immutability inherent in monads, with the added benefit that the computations are performed lazily and only when needed.

Difference between JDK Supplier and Lazy monad

Some striking differences between the monad we will be implementing versus the JDK Supplier functional interface are:

Supplier

Supports laziness by deferring the execution of the wrapped computation until the method get is called.
Does not support memoization, which means it does not cache the value once computed. This can lead to several issues in a multithreading application if not handled correctly:
- Expensive resources being acquired repeatedly.
- Computations meant to run once being executed multiple times.
- Potentially incorrect application state.
- The responsibility falls to the developer to ensure these issues do not occur.

Lazy monad

Supports laziness by deferring the execution of the wrapped computation until the method get is called.
Supports memoization. Upon the first call of the get method, the result of the wrapped computation is stored in memory. Further calls to get will always return the memoized value.
As a monad, it implements the methods map for applying a function to the type of the wrapped computation and flatMap to chain multiple computations involving the Lazy monad together.
The added benefit of map and flatMap is that they are lazy until get is called.
Every operation in the Lazy monad is immutable, which ensures thread safety and correctness in concurrent applications.

Implementation

Before proceeding with the implementation, we need to outline our goals. Our Lazy monad should achieve the following:

Laziness: Every operation, except for get, should defer the computation.
Transformation and Chaining: Provide map and flatMap methods to transform and chain computations in a functional programming style.
Memoization: Implement a get method that evaluates the computation once and memoizes the result.

Let’s proceed with the implementation:

public final class Lazy<T> {

  // Using a sentinel value to represent uninitialized state because null is a valid value.
  private static final Object UNINITIALIZED = new Object();

  private final Supplier<T> lazyValue;
  private final Object lock = new Object();
  private volatile Object evaluatedValue = UNINITIALIZED;

  private Lazy(Supplier<T> lazyValue) {
    this.lazyValue = lazyValue;
  }

  private Lazy(T value) {
    this.lazyValue = () -> value;
    this.evaluatedValue = value;
  }

  /**
   * Creates a new lazy value.
   *
   * @param supplier the supplier that provides the value
   * @param <T>      the type of the value
   * @return a new lazy value
   * @throws NullPointerException if the supplier is null
   */
  public static <T> Lazy<T> of(Supplier<T> supplier) {
    return new Lazy<>(Objects.requireNonNull(supplier));
  }

  /**
   * Creates a new lazy value with an already evaluated value. Useful when you want to memoize a
   * value that is already computed.
   *
   * @param value the value
   * @param <T>   the type of the value
   * @return a new lazy value
   */
  public static <T> Lazy<T> evaluated(T value) {
    return new Lazy<>(value);
  }

  /**
   * Gets the value. If the value has not been evaluated yet, it evaluates it and caches it.
   *
   * @return the value
   */
  @SuppressWarnings("unchecked")
  public T get() {
    Object value = evaluatedValue;
    if (value == UNINITIALIZED) {
      synchronized (lock) {
        value = evaluatedValue;
        if (value == UNINITIALIZED) {
          value = lazyValue.get();
          evaluatedValue = value;
        }
      }
    }
    return (T) value;
  }

  /**
   * Maps the value of this lazy instance to a new value.
   *
   * @param mapper the mapping function
   * @param <R>    the new type of the value
   * @return a new lazy instance with the mapped value
   */
  public <R> Lazy<R> map(Function<? super T, ? extends R> mapper) {
    return Lazy.of(() -> mapper.apply(this.get()));
  }

  /**
   * Flat maps the value of this lazy instance to a new lazy instance.
   * <b>It is susceptible to stack overflow if the function passed to it is not tail recursive.</b>
   *
   * @param mapper the mapping function
   * @param <R>    the new type of the value
   * @return a new lazy instance with the mapped value
   * @throws StackOverflowError if the function passed to it is not tail recursive
   */
  public <R> Lazy<R> flatMap(Function<? super T, Lazy<R>> mapper) {
    return Lazy.of(() -> mapper.apply(this.get()).get());
  }

}

Notes and considerations:

Deferred Computation:
- We have defined a method of which accepts a Supplier that wraps a deferred computation.
Memoization and Volatile Storage:
- We will store the evaluated computation in a volatile Object instance. Initially, this field is initialized with a sentinel value indicating that the computation has not been executed yet.
- The sentinel value also distinguishes cases where the computation returns null, which is a perfectly acceptable result.
Maintaining Laziness with map and flatMap:
- The methods map and flatMap retain the laziness of the computation by wrapping the initial Supplier in another Supplier.
- Care should be taken to avoid excessive recursion in flatMap operations, as this could lead to a stack overflow error.
Thread-Safe Evaluation with get:
- The get method uses a double-checked locking idiom to ensure that even if accessed by multiple threads before initialization, the computation is evaluated only once.

Examples

Defining a variable that when called will execute the expensive computation.

private final Lazy<Integer> lazy = Lazy.of(this::veryExpensiveComputation);

//complex statements here

return lazy.get(); //Finally execute the computation.

Chaining operations in a functional programming style.

Lazy.of(this::expensiveComputation)
  .map(value -> transformValueToValue2(value))
  .flatMap(value2 -> anotherExpensiveComputationReturningLazyAsReturnType)
  .get();

Ensuring that multiple threads can call the get method without recomputing the value.

Lazy<Integer> lazy = Lazy.of(this::veryExpensiveComputation);

ExecutorService executor = Executors.newFixedThreadPool(10);
executor.invokeAll(Collections.nCopies(10, () -> new Callable<Integer>() {
  @Override
  public Integer call() throws Exception {
    return lazy.get(); //All threads will get the same value without recomputation.
  }
}));

Combining other monads inside

Lazy<Either<Exception, Integer>> lazy = Lazy.of(() -> {
  try {
    return Either.right(veryExpensiveComputation());
  } catch (Exception e) {
    return Either.left(e);
  }
});

Conclusion

In this article, we have implemented a Lazy monad from scratch in Java. We have discussed the purpose of a Lazy monad, its use cases, and the differences between the JDK Supplier functional interface and our implementation. We have also outlined the goals of our implementation and provided examples of how to use the Lazy monad in practice.