One piece of optical hardware performs massively parallel AI calculations

Enlarge /. The output of two optical frequency combs shows the light that appears at evenly spaced wavelengths.

AI and machine learning techniques have become a focus of everything from cloud computing services to mobile phone manufacturers. Unfortunately, our existing processors are a poor match for the algorithms on which many of these techniques are based, partly because they require frequent round trips between the processor and memory. To overcome this bottleneck, researchers figured out how to perform computations in memory and designed chips with a bit of memory allocated to each processing unit.

Now two different research teams have found ways to perform calculations with light in such a way that both memory and calculations are merged and massive parallelism is possible. Despite the differences in implementation, the hardware developed by these teams has one thing in common: it enables the same hardware to perform different calculations with different light frequencies at the same time. While not yet reaching the level of performance of some dedicated processors, the approach can be easily scaled and implemented using on-chip hardware, thereby speeding up the process of using it as a dedicated co-processor.

A fine-toothed comb

The new work is based on hardware called a frequency comb. This technology gave some of its developers the 2005 Nobel Prize in Physics. While there is a lot of interesting physics behind how the combs work (which you can read more about here if you're curious), the result of that physics matters to us. There are several ways to create a frequency comb, but they all create the same thing: a beam of light made up of evenly distributed frequencies. A frequency comb in visible wavelengths could therefore consist of light with a wavelength of 500 nanometers, 510 nm, 520 nm etc.

The metaphor is a comb – a large number of evenly spaced teeth – but the underlying reality of the metaphor is that each of the teeth is a specific wavelength of light. Since frequency and wavelength are related, you can also view them as evenly spaced frequencies.

We have known for some time that it is possible to do calculations with light. The frequency comb provides a convenient means of making these calculations massively parallel. If the frequency comb described above were sent by hardware that manipulates visible wavelengths, each and every one of its component frequencies would perform the same calculation, making its operation essentially massively parallel. This is not particularly useful on its own unless you really want the results of a calculation to be confirmed.

advertising

However, with some types of frequency combs, each wavelength can be adjusted independently, increasing or decreasing the intensity of each tooth. This allows different calculations to be made with each tooth in the comb, while maintaining the massively parallel aspects.

One of the types of calculations that is relatively easy to map to light is matrix multiplication, which is widely used by some AI applications. In particular, it can be used to perform convolution, a mathematical operation that is part of the deep neural networks that have excelled in image recognition. While both papers use their optical hardware to manipulate images, they use very different approaches to get there. We start with the easier one, which is to be understood first.

Into the matrix

How do you actually perform image operations with light? The first step is to digitize the image. From there, details about each pixel can be encoded in the light intensity at certain wavelengths of the frequency comb. These wavelengths are then sent into a square grid of phase change materials. The phase change material performs operations on light because it absorbs different amounts of light depending on how ordered or disordered the material is. Depending on the path the light takes through the grating, different amounts will be absorbed, with the final intensity being the readout of the operation, which is essentially vector multiplication accumulation.

It is crucial that the grating can perform this operation simultaneously on different wavelengths. In this way, different teeth of the comb can be fed into the hardware at the same time. And because the phase change material is reconfigurable, it can be reconfigured to perform various operations at will. As is standard for this type of demonstration process, the researchers tuned the hardware to use it for a series of handwritten digits that have become standard in the field. An accuracy of over 95 percent could be achieved, which is considered successful.

For the demonstration, the various hardware parts – the laser, the hardware that converts it into a frequency comb, the phase change memory, and the photon detectors – were all on different chips. However, nothing prevents them from being integrated on a single chip so that they can be used as an optical co-processor for AI tasks.

Two major limits to its operation are the number of teeth in the frequency comb and the size of the phase change material grid, and there are ways to increase both. A little more difficult to handle are the limits that result from how fast the optical hardware can work. Since there are ways to deal with this, the current operation of the hardware should be viewed as the lower limit. But even as a lower bound, it's pretty impressive and can run at 3.8 trillion operations per second.

advertising

While there are already specialized AI co-processors out there, they work on the same principles as regular processors, so heat and power become a problem for them too. A major potential benefit is that the heating and power issues here focus almost entirely on the laser light source. As long as these issues can be resolved, the device can be operated at full tilt with no additional concern about these issues.

A question of time

It should be relatively easy to imagine a bundle of photons rotating around a grid of semi-reflective materials. The other paper on the same subject relies on a very difficult and less appreciated behavior of light: when light passes through a material with strong internal refraction, the speed at which the light moves depends on its wavelength.

To take advantage of this, the researchers encode information as vectors into some teeth on the frequency comb. This light is then sent over an optical cable that creates different delays at different wavelengths. However, the detector on the other side of the cable only accepts input during a certain time window. If the delay moves some parts of the input vectors out of this time window, they are not counted in the output. In essence, this performs a mathematical operation called convolution that relates the frequency at which the data was encoded to the weight that the detector gives based on when it arrived.

And again, convolution is critical to the deep neural networks used for image classification. Here, too, many turns can be made in parallel, since the optical cabling can carry a wide range of wavelengths. As a result, the setup used in the paper was able to achieve 11.3 trillion operations per second, even though it performed slightly worse on the digit recognition test with an accuracy of 88 percent.

While the individual operations are quick, the setup absolutely needs the delay to function properly, which means it can never deliver the kind of instant response that some applications require. However, it can be operated with standard optical telecommunication devices. If all of the tapes used in modern telecommunications hardware were used for operations, over 400 operations could be performed in parallel. And that's before things like the polarization of light are used to convey information that could bring performance to the quadrillion operations per second range.

Nature, 2010. DOI: 10.1038 / s41586-020-03070-1, 10.1038 / s41586-020-03063-0 (About DOIs).