Shattering the Memory Wall: Why Photonic Interconnects Are the Future of Agentic AI

If you've spent time optimizing backend systems, you know the feeling. You profile a service, expecting to find slow business logic, and find the code is fine. The thread is just sitting there, waiting on a database call, a disk read, or a cache miss. Much of performance engineering comes down to one idea: the work is rarely slow, but getting the data to the work is.

That idea is now defining the future of AI hardware. We haven't run out of compute. We've started to run out of ways to feed it, and the culprit is a material we've relied on for over a century: copper.

This article explains why, starting from the basics. By the end, you'll see why companies like Volantis are betting on light instead of electricity to build the next generation of AI infrastructure.

---

First, the Basics: What a Processor Actually Does All Day

A GPU does arithmetic, mostly multiplying and adding enormous grids of numbers (matrices). That's the core of how an LLM generates a word. But arithmetic needs numbers, and those numbers live in memory.

Think of a chef in a kitchen. The chef can chop incredibly fast, but if the ingredients are in a warehouse across town, the chef spends most of the day standing around. How fast the chef chops matters far less than how fast ingredients arrive.

Two terms matter here, and people often mix them up:

• Latency is how long a single piece of data takes to arrive after you ask for it. It's the time for one trip to the pantry.
• Bandwidth is how much data can flow per second. It's how many ingredients the delivery truck carries per trip, and how many trucks run at once.

AI workloads are hungry for both. A modern model has billions of parameters (the "weights" it learned during training), and generating a single token means streaming a huge share of those weights from memory into the compute units. When the memory system can't keep up, the GPU sits idle, burning power and doing nothing. Engineers call this being memory-bound.

This gap between how fast processors compute and how fast memory can supply data has grown for decades. It's called the Memory Wall.

Why Memory Has to Sit So Close

To fight the Memory Wall, hardware designers use High Bandwidth Memory (HBM): stacks of memory chips placed right next to the GPU die, on the same package, connected by thousands of very short wires.

The proximity is no accident. It's forced by physics. Data inside a chip travels as electrical signals, and moving a signal along a copper wire gets harder as the wire gets longer and the signal gets faster:

1. Signal degradation. At very high frequencies, copper traces blur and weaken a signal. The faster you push bits, the shorter the distance they can cleanly travel before the receiver can't tell a 1 from a 0.
2. Energy cost. Pushing electrons through a resistive wire wastes energy as heat, and the penalty grows with distance and speed. Moving data often costs more energy than computing on it.
3. Compensation circuitry. To clean up weakened signals, engineers add extra circuits to boost, equalize, and re-time them. That adds latency, power, and chip area.

So the rule of thumb is: the farther data must travel electrically, the more it costs. That's why HBM hugs the processor, and why only so much of it fits. There's a small ring of real estate around the compute die, and it fills up fast.

Here's the backend analogy. Your CPU has L1, L2, and L3 caches that are tiny but fast, then RAM that's bigger but slower, then disk, then network. The hierarchy exists because you can't have capacity and speed at once. AI accelerators face the same trade-off: a little very fast memory nearby, and much more memory that is far away and painfully slow to reach.

When the Memory Wall Meets Agentic AI

For a simple chatbot, the wall is manageable: one prompt, one response, and a model that fits in a known amount of memory.

Agentic AI is different. An agent doesn't answer once and stop. It runs in a loop:

• It reads the goal and its current state.
• It plans a step, perhaps calling a tool or searching a knowledge base.
• It reads the result and updates its understanding.
• It repeats, often dozens or hundreds of times.

Each loop makes memory demands that a single chat turn doesn't:

• A huge context window. The agent must keep a long history of what it has done and seen. That history is stored as a KV cache, which grows with every step and can consume tens of gigabytes.
• Retrieval (RAG). Retrieval-Augmented Generation means the agent looks up relevant information from a large store of vector embeddings, numeric fingerprints of documents, before answering. That's a lot of data to search quickly.
• Model weights. Frontier models are enormous and must be reachable the whole time.

When all of this doesn't fit in the fast memory beside the processor, the system has to reach out over slower, longer connections. That's where it stalls. In software terms, it's a blocking I/O call in the middle of your hot path. The thread isn't broken. It just waits, and in an agent loop those waits compound with every iteration.

The usual workaround is to split the model across many GPUs. But now the GPUs have to talk to each other over more electrical links, and the bottleneck moves rather than disappears.

Enter Light: How Silicon Photonics Works

Silicon photonics replaces some of those electrical wires with beams of light carried through microscopic channels etched into silicon. The main pieces are:

• A laser produces the light, which acts as the carrier.
• A modulator switches the light on and off (or alters it) very quickly, encoding bits onto it. It's the equivalent of flipping a switch billions of times per second.
• A waveguide is a tiny channel that guides the light, like an optical fiber shrunk onto a chip or board.
• A photodetector at the other end converts the light back into an electrical signal.

Why is this better over distance? Light in a good waveguide loses very little signal over meters, where a fast electrical signal would struggle over centimeters. Light also lets you send many independent data streams down the same waveguide at once, using different colors (wavelengths). Electrical wires can't do that, so optical links can carry far more bandwidth through the same physical space.

It also changes the energy picture. Moving a bit over a long electrical link can cost several picojoules (a picojoule is a trillionth of a joule). Photonic designs target around one picojoule per bit, even over much longer distances. At the scale of a data center moving unfathomable quantities of data, that difference decides whether a rack stays cool or melts.

One honest note: light is not instantaneous. It still takes roughly five nanoseconds to travel a meter of fiber. The win is that it stays fast, clean, and cheap in energy over distances where copper gives up, not that latency vanishes.

What Changes in the Architecture

Once distance stops being a severe penalty, the design can be rethought:

1. Decoupled memory. Memory no longer has to be glued to the processor. It can sit elsewhere on the board or in the rack, connected by optical links.

2. Rack-scale memory pools. Instead of each GPU owning a small private slice of memory, a compute unit can reach a huge shared pool, potentially hundreds of memory chips and terabytes of capacity. This is the hardware version of moving from a local disk to a shared network store, except that the access is designed to be fast.

3. Cooler, denser systems. If moving data costs far less energy, there's less heat to remove, and more hardware fits in the same space and power budget.

4. A system that behaves as one. Rather than a cluster of chips constantly waiting on each other, the whole rack can start to act like one large machine with one large memory.

Volantis AI: Betting on the Plumbing

This idea is moving from research papers into products. Volantis, a San Mateo-based semiconductor startup, recently raised an $88 million Series A to build photonic motherboards.

Its strategy is notable for what it doesn't do. Volantis isn't trying to out-engineer Nvidia at matrix multiplication. It's targeting the I/O layer, the plumbing between compute and memory. Using micro-VCSEL lasers (tiny lasers that emit light straight up from the chip surface, which makes them compact and easy to mass-produce) and proprietary optical waveguides, it is building rack-scale systems with these target specifications:

• Pooling over 100 TB of memory per chip
• Bandwidth above 30 TB/s
• Support for frontier models exceeding 10 trillion parameters
• A projected up to 15x better performance per dollar than current-generation hardware

These are the company's own claims and projections, so treat them as goals until independent benchmarks confirm them.

The Hard Parts

Photonics isn't free magic, and a fair account should say so:

• Manufacturing. Making lasers, waveguides, and electronics work together at scale, with high yield and low cost, is difficult.
• Reliability. Lasers can fail, and a data center can't tolerate frequent replacements.
• Packaging and alignment. Getting light cleanly on and off a chip takes extreme precision.
• Ecosystem. Software, standards, and existing infrastructure were built around electrical systems, and the industry doesn't switch overnight.

These are why photonics has been "five years away" for a long time. What has changed is demand: AI workloads have made the cost of not solving them large enough to justify the investment.

Why Agents Make This Urgent

Return to the agent loop. An agent needs its working memory, its vector databases, and its model weights all reachable at once, and it needs them again on every iteration. If each step triggers a slow trip across electrical buses, the agent's "thinking" slows down in direct proportion.

A photonic memory fabric lets the hardware keep everything in a state of immediate accessibility. For the agent, that's the difference between a developer whose reference material sits open on the desk and one who has to drive to the library for every question.

The Road Ahead

We've spent years making software smarter at hiding I/O latency: async runtimes, caches, prefetching, batching. Those techniques are valuable, but they only work inside the limits the hardware sets. When the physical medium is the bottleneck, no clever scheduling can move data faster than the wire allows.

Copper has carried computing remarkably far. The next phase of AI won't be defined only by better algorithms or more parameters. It will also be defined by how efficiently we can move data, and that increasingly means moving it with light.