3.2.1  Pipeline ADCs

Pipeline ADCs [4] have a clocked topology with several operation stages. Each stage resolves a certain number of bits of the analog-to-digital conversion. The output of each stage is connected to the next. The m-most significant bits are computed by the first stage, and the rest of the bits are computed by the successive ones. There is the same delay between the output of each stage. Figure 3.1 shows the schematics of a pipeline ADC with m-bit conversion stages. Each stage resolves m-bits and is made up of one m-bit ADC and one m-bit digital-to-analog converter (DAC). Then, the remaining stage output is amplified with a gain A = 2^m and goes to the next stage. Because the bits from each stage are determined at different clock cycles, all the bits corresponding to the same sample have to be time-aligned with shift registers. This imposes area requirements and circuit complexity that limits the usage of these converters with modern column-parallel implementations. They have been employed as global ADCs to read out imager pixel outputs [5]. Its main advantage is high conversion speed. Clock cycles of N_bits are required to digitize the input voltage. However, in every clock cycle, the same conversion bits are available from the previous input. The pixel output could be changed after reading the most significant bits of the first conversion stage, adding the possibility of implementing pipeline operation with the pixel readout. Thus, the equivalent pixel conversion speed would be only one clock cycle.

3.2.2  Slope ADCs

Slope ADCs (also known as ramp ADCs) are the most extended converters for image sensors with column-parallel readout topology. Their architecture is rather simple (see Figure 3.2). They have a ramp generator, a comparator, a counter, and an output memory. Initially, the ramp generator and the counter are reset. When the conversion starts, the counter and the ramp generator are activated. If the slope voltage exceeds the input voltage, the counter is stopped, and its outputs are latched. Their advantages are simplicity, low area requirements (they can be lay out with a fine pixel pitch), and acceptable FPN. Their low area requirements are a great advantage for modern image sensors with a large number of pixels. They can even be embedded inside individual pixels [6] leading to high-speed image sensors.

Figure 3.1   Pipeline ADC block diagram.

Figure 3.2   Ramp ADC block diagram and principle of operation. CLK, counter clock; CNT_Enb, counter outputs enable.

The main disadvantage of slope ADCs is their conversion speed since of clock cycles are required to carry out a conversion. Thus, they are more suitable for ADCs with a low bit resolution. To overcome this limitation, several authors propose multiramp ADCs. The input signal is compared to a fast slope to determine the coarse bits of the input signal. Then, a slope with an offset that corresponds to the course of the input signal is activated to determine the less significant bits of the input signal.

There are also some other issues with this architecture: (a) misalignment between coarse and fine conversion ramps and (b) circuit complexity that grows exponentially with the number of coarse-bit conversion. FPN related to ramp ADCs is mainly due to the mismatch of the ramp generators. Ramp generators commonly use capacitors. To improve this limitation, the ramp generators’ capacitance is increased. This imposes area requirements to the ADC layout.

3.2.3  SAR ADCs

SAR ADCs overcome the speed limitations of ramp ADCs. SAR ADCs require only N_bits clock cycles to perform an ADC conversion. The main limitations of SAR ADCs are area and power consumption. Their principle of operation is as follows (see Figure 3.3): initially, the input analog signal is sampled and held. Then, it is compared to a reference voltage value. The comparison result sets the value of the most significant bit (MSB). Then, the reference voltage is changed, and the comparison is repeated N_bits—one time. The control signal end of cell indicates the end of the analog-to-digital conversion. A DAC and digital logic are employed to set the voltage reference values. This imposes expensive area requirements that makes it challenging to implement a SAR for each column. To amend this limitation, it would be possible to share an ADC by a group of pixels. However, in that case, the readout speed requirements would increase. Thus, there is a trade-off between area requirements and speed. Another disadvantage of this kind of converters is the mismatch that is introduced by the DAC. Some authors [7] try to overcome it by calibrating the reference voltage of different DACs. The penalties are more area consumption and circuit complexity. In any case, the DAC design should take into account mismatch considerations to reduce FPN.

Figure 3.3   SAR ADC block diagram.

3.2.4  Cyclic ADCs

The principle of operation of cyclic ADCs is similar to SAR ADCs. They only need N_bits, clock cycles to operate. The main difference is that the reference voltage, V_ref, does not need to be generated during each iteration. Hence, a DAC is not necessary. They employ switched-capacitor (SC) amplifiers, which amplify the difference between the input signal and the reference voltage (see Figure 3.4).

The input signal is changed after each operation cycle. It will be equal to the remaining voltage at the output of the amplifier. After each operation cycle, the remaining input signal is compared to V_ref to generate the next MSB. Depending on the value of the MSB, either V_ref or zero will be subtracted from the remaining input voltage in the next cycle.

The voltage at the output of the amplifier V_resout can be expressed as

3.1 $$V_{\text{resout}}\left(K\right)=A\cdot V_{\text{resin}}\left(K\right)-d_k\cdot V_{\text{ref}}$$

where k is the number of the conversion cycle.

Variations of the amplifiers gain can lead to high FPN that can be unacceptable for high-quality image sensors. This imposes a good capacitor matching (more area requirements) and higher currents for the amplifier biasing (more power consumption). FPN and area are the main limitation of cyclic converters for image sensors.

Figure 3.4   Cyclic ADC block diagram.

Figure 3.5   ΣΔ ADC block diagram.

3.2.5  ΣΔ ADCs

ΣΔ ADCs (see Figure 3.5) have traditionally been used as global ADCs with imagers with low speed and low FPN requirements. They consist of a ΣΔ modulator followed by a digital decimation filter. The modulator has an integrator and a comparator with a feedback loop that contains a 1-bit DAC. The DAC just connects the comparator input to a positive or a negative reference voltage. A clock is necessary to provide the correct timing for the modulator and the digital filter. These kinds of converters suppress the input noise by oversampling. The voltage variations of the input signal are tracked quickly by ΣΔ converters. However, for image sensors with column-parallel readout, it is not necessary to track fast variations of the input signal that can be considered static during the readout process. For this reason, some authors design incremental ΣΔ converters that are optimized to read quasi-static DC signals and negligible offset [8–10].

Their main advantages are low FPN, good resolution, and speed. Their disadvantages are the area requirements, circuit complexity for column-parallel implementation, and their continuous operation mode that can lead to an unnecessary power consumption when they are employed to read out static image sensors’ outputs. As it will be discussed in Section 3.6, some of these drawbacks can be solved by adapting the design of ΣΔ converters to the specific requirements of image sensors leading to a very competitive emerging generation of converters.

3.3.1  Global ADCs

The traditional approach was to use a single ADC per sensor. All the pixel outputs were time-multiplexed to the input of a global ADC, as shown in Figure 3.6a, to provide a digital output. The analog-to-digital conversion frequency of an entire frame is given by the following equation [2]:

Figure 3.6   (a) Readout scheme with a global ADC. (b) Readout scheme with global internal buffer for an external ADC. (c) ADCs in column-parallel distribution.

3.2 $$f_{\text{conv}}={\left[M\cdot N\cdot \left({\tau }_{\text{ADC}}+{\tau }_{\text{RO}}\cdot \frac{n_{\text{bits}}}{n_{\text{parallel}}}\right)\right]}^{-1}$$

where M and N are the number of pixels per row and column, respectively, τ_ADC is the ADC sampling time (τ_ADC = 1/f_s), τ_RO is the time that is required to send 1 bit out of the chip, which depends on the master clock frequency, n_bits is the number of bits per sample, and n_parallel is the number of parallel digital outputs. The ADC was usually integrated on the chip to maximize the frame rate and reduce the FPN. Global ADCs were usually designed to maximize the readout speed. Designers were not very concerned about the area requirements or the power consumption of the ADC. The pixel array area and power consumption were usually much higher than the ADC area and power consumption.

Another approach with a global ADC was to provide an analog readout. The ADC was placed off-chip (see Figure 3.6b). In this case, pixel outputs were first time-multiplexed and then connected to the input of a global internal analog buffer. Analog intermediate memories could also be used to increase the frame rate of the sensor [11]. The internal buffer should be fast enough to guarantee the desired frame rate. It should also be able to provide enough output current to charge or discharge on time the high capacitive load that is introduced by the output pad and the long metal lines.

3.3.2  Column-Parallel ADCs

The demand of commercial sensors with high resolution and high frame rates requires to significantly reduce the time that is dedicated to read out the pixel outputs. Under these requirements, global ADCs are not practical and have been deprecated during the previous years. Obviously, by increasing the number of ADCs working in parallel with different groups of pixels, the frame readout time will be reduced.

For simplicity, by placing one ADC per column (see Figure 3.6c), all the pixels of one row can be sampled and read out simultaneously. This approach has a twofold benefit: the frame readout speed is increased significantly, and the requirements for the conversion time are less restrictive. The frequency of the frame analog-to-digital conversion is almost M times faster:

3.3 $$f_{\text{conv}}={\left[M\cdot N\cdot \left(\frac{{\tau }_{\text{ADC}}}{M}+{\tau }_{\text{RO}}\cdot \frac{n_{\text{bits}}}{n_{\text{parallel}}}\right)\right]}^{-1}$$

The main challenges are the area requirements for the ADC. The ADC width should be lower than the pixel pitch. The power consumption should be low because the number of ADCs will be increased significantly. Some authors [12] place one ADC per column alternatively on the top and bottom of the array. Thus, the ADC pitch can be doubled relaxing the ADC area design requirements and achieving the same readout speed than with the column-parallel implementation of Figure 3.6.

It would be even possible to divide the sensor into two blocks and place an ADC per column at the top and bottom of the sensor. The two blocks could be read out in parallel employing a correlated double sampling approach, almost doubling the conversion frequency. This technique is more attractive with pixels with large pitch, typically biomedical arrays with low resolution, where the ADC area and the power requirements are less exigent.

Another issue to take into account with multiple ADCs per sensor is the output data flow. We should have circuitry to store and send out all the ADC outputs. External devices connected to our sensor should also be able to digest the output bit stream.

3.3.3  Pixel-Level Analog-to-Digital Conversion

Some authors [6,13,14] propose digital pixel sensors (DPSs) with integrated ADCs into each pixel. The DPS performance takes advantage of the CMOS scaling-down properties. The idea is simple: each pixel has a dedicated ADC and provides an independent digital output, considerably increasing the readout speed. In this case, the conversion frequency is given by

3.4 $$f_{\text{conv}}={\left[{\tau }_{\text{ADC}}+{\tau }_{\text{RO}}\cdot M\cdot N\cdot \frac{n_{\text{bits}}}{n_{\text{parallel}}}\right]}^{-1}$$

Although the conversion speed is higher, there are important drawbacks. This method reduces the pixels’ fill factor. It also requires massive parallel circuitry that is capable of reading large amounts of data.

Figure 3.7 compares f_conv for the three different readout topologies for different array sizes (M · N). We assume for the computation that, M = N, τ_ADC = 1 μs, τ_RO = 10 ns, and n_bits = n_parallel = 12. These values correspond to standard modern ADC features (see Section 3.5 for more details). The value f_conv is always higher using multiple ADCs. For arrays with a low number of pixels, the DPS topologies achieve the highest speed. However, for large-pixel arrays, this topology offers the same performance than the column-parallel topology. For this reason, DPSs are usually aimed for high-speed image sensor applications with low-resolution arrays.

Figure 3.7   Frequency of conversion for different ADC topologies: (*) global ADCs, (♢) column-parallel ADCs, and (◻) pixel-level ADCs.

Figure 3.8   Diagram of pixel with an internal ramp ADC and n-bit digital memory.

Figure 3.8 shows a DPS with an integrated n-bit analog converter. The pixel has an n-bit ramp converter and a memory to save its outputs. To reduce the pixel area, some circuitry of the ADCs typically can be shared by the different pixels. In this example, the ramp generator and the counters are shared by all the pixels. Each pixel requires a comparator and a memory to perform the pixel-level analog-to-digital conversion.

3.3.4  Future Integrations

The development of 3D technologies with stacked and interconnected dies opens new possibilities to increase the readout speed without reducing the fill factor. One entire die (tier) could be dedicated for the ADC design. Ideally, the photoactive area could be placed on the top level, and an ADC per pixel could be placed below it (see Figure 3.9). Kiyoyama et al. recently reported a very interesting ADC for 3D integration [3]. Each pixel block was connected to a dedicated ADC on the bottom tier with a pitch of 100 × 100 μm². However, one dedicated ADC per pixel is not feasible with photodiodes with a fine pitch. In this case, it would not be possible to fit an ADC or a processing circuitry below the photodiodes. Thus, the pitch of the processing circuitry will be higher than the photodiode pitch. This implies design challenges to face: (a) groups of photodiodes should share processing circuitry, and (b) they should be multiplexed. Figure 3.10 illustrates a possible implementation with multidistributed processing units (PUs). The pitch of the PUs is bigger than the photodiode pitch. In such topology, f_conv is given as

3.5 $$f_{\text{conv}}={\left[M\cdot N\cdot \left(\frac{{\tau }_{\text{ADC}}}{N_{\text{adc}}}+{\tau }_{\text{RO}}\cdot \frac{n_{\text{bits}}}{n_{\text{parallel}}}\right)\right]}^{-1}$$

where N_adc is the number of ADCs on the bottom tier.

Figure 3.9   Diagram of a 3D stacked system with dedicated ADC, memory, and processing elements per pixel. (Adapted from K. Kiyoyama et al., A very low area ADC for 3D stacked CMOS image processing system, IEEE International 3D Systems Integration Conference, 3DIC, pp. 1–4, 2011.)

Figure 3.10   Possible future 3D integration. Top layer is dedicated to photodiodes. Bottom layer is dedicated to PUs.

3.4.1  ADC Resolution

Figure 3.11 displays an image that is digitized with different numbers of bits. With 4 bits, we clearly notice a loss of image quality for using an insufficient number of bits. With six or more bits, there is not a significative improvement on the image quality. However, a much better resolution is needed to properly capture dark images, when the converter’s equivalent input noise is similar to pixel noise. We can wonder how many gray levels our eyes can visualize [15]. There is no clear answer, but several studies have converged that they are roughly 450 under certain illumination conditions. The Digital Imaging and Communications in Medicine (DICOM) standard sets a maximum of 450 gray levels for display representation, for instance. This means that 9-bit resolution is enough for an ADC conversion of the pixel outputs. This is not a very restrictive ADC resolution requirement, taking into account the average bit resolution of the currently reported ADCs. ADCs for image sensors usually employ 10 or more bits.

Figure 3.11   Number of bits’ effect on image quality. From left to right: 4-, 6-, and 8-bit precision.

The photon shot noise of image sensors can even be exploited to relax the requirements of quantization noise that is introduced by ADCs [16]. Photon shot noise is due to the temporal variations of the number of incident photons. It becomes the dominant noise source with high levels of illumination. Figure 3.12 shows the dependence of a sensor output and various noise sources with illumination. The output has a linear dependence with the illumination level until it saturates at level N_sat, which corresponds to the full well capacity of the sensor. Almost all the noise sources are independent of illumination (1/f noise, thermal, etc.) and set a noise floor that limits the sensor’s dynamic range of operation. However, the photon shot noise depends on the input signal level N_sig (number of electrons generated by the photodiode during the integration period) as follows:

3.6 $$e_{\text{phs}}(N_{\text{sig}})=\sqrt{N_{\text{sig}}}$$

Photon noise is dominant with bright illumination. We can see that the ADC in this region has less quantization noise than required. Hence, it would be possible to relax the ADC requirements that are related to resolution to reduce the power consumption or increase the speed, for instance.

We can define a quality parameter r that is given by the ratio of the quantization noise and the photon noise due to the signal as

3.7 $$r=\frac{e_{\text{qns}}(k)}{e_{\text{phs}}(N_{\text{sig}})}$$

The quantization noise e_qns(k) depends on the quantization step size e_lsb and the number of quantization steps k, as follows:

3.8 $$e_{\text{qns}}(k)=k\cdot \frac{e_{\text{lsb}}}{\sqrt{12}}$$

Figure 3.12   Dependence of the sensor output and the main noise sources with illumination. (Adapted from M. F. Snoeij et al., IEEE Journal of Solid-State Circuits, vol. 42, no. 12, 2007.)

For optimal ADC design, the input range of the ADC should be matched to the maximum output swing of the sensor N_sat:

3.9 $$N_{\text{sat}}=2^n\cdot e_{\text{lsb}}$$

And, combining Equations 3.6 through 3.9, we can compute the input signal levels at which we can double the quantization noise, i.e., to reduce the ADC resolution to 1 bit, maintaining the ratio between the quantization and the photon shot noise constant:

3.10 $$N_{\text{sig}}={\left(\frac{k\cdot N_{\text{sat}}}{2^n\cdot r\cdot \sqrt{12}}\right)}^2$$

Figure 3.13 illustrates how the photon shot noise can be exploited to increase the quantization levels. An elegant design that exploits this property with multiramp ADCs was described by Snoeij et al. [15].

3.4.2  Random Noise

Figure 3.13   Illustration of how shot noise can be exploited to relax the ADC quantization noise specifications. (Adapted from M. F. Snoeij et al., IEEE Journal of Solid-State Circuits, vol. 42, no. 12, 2007.)

Another feature to take into account is the random noise that is introduced by the converter. It can be expressed in volts, but image sensor designers usually express it in electrons, which is related to the full well capacity of the vision sensor. Random noise provokes column FPN. This is a mismatch that is introduced by the different column ADCs in a column-parallel readout topology. If the different ADCs connected to each column introduce mismatch, we will perceive column variations of the gray levels when the array is illuminated with uniform illumination. As mentioned in Section 3.1, humans can detect a 0.5% change in mean intensity [1]. Therefore, the FPN introduced by an ADC is a very important parameter to take into account for high-quality image sensors. Figure 3.14 illustrates the effect of FPN in a column-parallel readout topology. One captured image with three different levels of FPN is shown. On the right, the degradation provoked by FPN on the image is clearly displayed. We observe pixel columns with different gray levels in regions with uniform illumination. This undesirable effect can be improved if the ADCs do not always read the output of the same column in a column-parallel implementation. If the ADC reads out a column that is shifted every time that a different row is selected, the column FPN will be improved.

Figure 3.14   FPN’s effect on image quality.

3.4.3  Area

The area requirements are restrictive for image sensors. If we employ one ADC per column, the ADC pitch must be equal to or lower than the pixel pitch. This limits in many cases the nature of the ADC that we can use. Nowadays, we can find very fine pixel pitches below 1 μm in commercial image sensors. The ADC layout can be very tricky and complicated. Capacitors require a large area to be fitted in a rectangle width that is lower than the pixel pitch. Metal-insulator-metal (MiM), metal-oxide-metal (MoM), and polycapacitors are commonly used to implement SC circuits. Depending on the technology, we can find design rules that limit the pixel pitch.

3.4.4  Speed

The speed of the analog-to-digital conversion is an important parameter to consider. The ADC sampling frequency, f_s = 1/τ_ADC, should be higher than

3.11 $$f_{\text{s}}\gg \frac{1}{\frac{N_{\text{ADC}}}{\text{FR}\cdot N_{\text{pixels}}}-\frac{{\tau }_{\text{RO}}\cdot N_{\text{ADC}}}{n_{\text{bits}}}}$$

where FR is the target frame rate, N_ADC is the number of ADCs that are shared by all the pixels, τ_RO is the amount of time to send 1 bit out the chip, and n_parallel is the number of digital outputs. Nowadays, for high-resolution image sensors, usually, N_ADC > 1. The maximum speed can be achieved when an ADC per pixel is implemented. In that case, all the pixels of the array can be read out simultaneously. Thus, there is a trade-off between readout speed, pixel complexity, and output sensor throughput.

3.4.5  Power Consumption

Since most image sensors are used in mobile devices, power consumption has become an important drawback. Nowadays, the dominant component of power consumption in CMOS image sensors with column-parallel ADCs is analog-to-digital conversion followed by output readout [17,18]. Thus, there is a trade-off between conversion speed and consumption. Power consumption in CMOS image sensors increases at least linearly in resolution and frame rate.

3.5.1  ADC Efficiency and Design Considerations

Looking at the ADCs for image sensors reported in Table 3.1, we can wonder how efficient they are in terms of energy consumption. There is a theoretic energy that is bound for the sampling energy that is imposed by thermal noise [19]:

3.12 $$E_{\text{min}}=8\cdot k\cdot T\cdot \text{SNR}=8\cdot k\cdot T\cdot 1.5\cdot 2^{N_{\text{bits}}}$$

In Figure 3.15, we have plotted the energy per sample of each converter that is normalized by E_min. Murmann [20] established a limit of E_s/E_min = 100 for the most efficient current high signal/noise ratio converters, for which energy consumption is just limited by thermal noise (when signal-to-noise and distortion ratio (SINADR) > 60 dB). Looking at Figure 3.15, we see that the most energy-efficient converters for image sensors are still two orders of magnitude above the Murmann’s energy limit for modern ADCs. SAR and cyclic converters are more efficient. Slope converters are less efficient and appear on the top of the plot far away from the Murmann’s bound. Therefore, the choice of the ADC architecture has an influence on the energy efficiency that should be taken into account.

Figure 3.16 displays the dependency between the ADC resolution and the energy consumption per sample E_s = P/f_s. There is an exponential dependency between them. Increasing the bit resolution provokes an exponential increment of the energy consumption. We also have to consider that the total ADC power consumption will be multiplied by the number of columns in a column-parallel implementation. As it was discussed in Section 3.4.1, increasing the ADC resolution does not necessarily improve the image quality beyond nine effective bits. Therefore, limiting the number of effective bits to 10 will save a lot of power.

3.5.2  ADC Comparison

The imager’s requirements can be very different depending on the application: surveillance, industrial inspection, video, microscopy, photography, etc. Some applications require not only low image quality but also low power consumption or high-speed operation (surveillance or industrial inspection), for instance. Others require high image quality (fine pixel pitch) and are less restrictive with operation speed or power consumption, i.e., high-quality imaging. Therefore, ADCs for image sensors should be compared taking into account the scope of application of the imager whose outputs have to be converted. There are specific FoMs to compare ADCs (International Solid-State Circuits Conference’s FoM, Walden’s FoM, Emmert’s FoM [34], Jonsson’s FoM [35], etc.). However, there are no FoMs for ADCs for image sensors. In order to compare the performance of the different ADCs for image sensors, we define the following FoM:

Figure 3.15   ADC conversion energy normalized to Emin and Murmann’s energy bound.

Figure 3.16   Energy per sample consumption versus ADC bit resolution. We have plotted in red an exponential data fitting.

3.13 $$\text{FoM}=\frac{N_{\text{s}}\cdot {\text{pitch}}^2\cdot P}{2^{N_{\text{bits}}\cdot }{f}_{\text{s}}}$$

The FoM was written in the lower-is-better form to simplify comparison. It rewards the number of bits, N_bits, and the speed sampling frequency, f_s. It penalizes the random noise, N_s, of the ADC (expressed in V_rms), the power consumption, P, and the minimum pixel width that can be read out (pitch). As it was discussed in Section 3.4.3, maybe the major design constraint with modern ADC topologies for image sensors is the ADC width (pitch). The required value of the sampling frequency, f_s, should be calculated taking into account the ADC topology, as explained in Section 3.3. The power consumption should be multiplied by the total number of ADCs.

We have compared the column-parallel ADCs of Table 3.1 using the specific FoM that was just defined. Table 3.2 summarizes the FoM values of all the sensors that were reported in Table 3.1. To make the comparison, we assigned the worst reported parameters to the nondisclosed (ND) values of the ADCs that were referenced in Table 3.1.

According to this FoM, ramp ADCs [21,24,25] perform worse than SAR or cyclic converters. They are not suitable for high-speed image sensors. For instance, the ramp ADC reported by [25] can operate with conversion rates up to 0.13 MHz, but it needs very fast internal clocks at 2.37 GHz. However, they can be integrated with very fine pitches, and this makes them popular. The SAR ADCs [23,29] offer good performance, with pitches of 4.2 and 2.25 μm, respectively. ΣΔ ADC [18,26] offers a good compromise between all the parameters that we are considering. Finally, cyclic ADCs [17,22,28] score very well with our FoM, but they are limited to pixels with higher pitches than other converters (above 5 μm).