Atmospheric particulate matter (APM)
that is
inclusively known as
aerosols, is the
main
cause of scattering, absorption, reflection, and
refraction of light in the medium. Photographs of a natural scene under high
environmental pollution
are known to
reduce radiance due to
attenuated direct scene transmission and cumulative additive scattered
surrounding light familiar as airlight or veiling light [6-12, 18, 20, 30]. The
Airlight dominates the distant object scene radiance due to attenuation of
direct transmission and diminishes to zero. Ultimately, low contrast and a
whitish veil cover the image entirely.
To address the
problem, single
image prior based visibility improvement algorithms
are
used that
dominate in the dehazing research domain. These techniques hinge
on a physical-based image formation model combining convexly with the direct
scene transmission and airlight. The majority of image dehazing methods recover
scene radiance by replacing the layer of haze
and rely
on a physical image
formation model [1, 2, 10, 11]
for this.
In section 3, this mechanism has been described
elaborately with the coefficients of the
linear combination
that
represents
the scene transmission (visibility) at each image pixel. A pixel
in an RGB image represents
the
four unknown parameters- a) the scene radiance at each R, G, B
colour channel and b) the transmission value. Whereas, the input captured image
supplies only three constraints, the intensity of each R, G, B channel. To
rectify this indeterminacy, most of the renowned methods rely on additional
information about the scene- a) multiple images photographed in dissimilar
weather conditions [36], b) polarization angles [35], and c) knowledge of the
scene geometry [39]. In recent times, methods have been developed to replace
an
additional input
requirements. This has been earned by penalizing the physical model
with
such properties as
maximal contrast [38], or by complimentary assumptions on hazy
atmosphere. In Fattal [20], the indeterminacy in the computation of
transmission has been relaxed by eliminating scattering light. A solution has
been formulated to resolve the ambiguity locally with no correlation between
the transmission and surface shading functions in slow varying regions. He et
al. [18] efficiently stated that there is a lowest intensity patch of pixels
among the pixels in the dark (low-intensity) colour channel, known as Dark
Channel Prior (DCP). Such pixels are found across the entire image.
Transmission estimation is
known to be
inaccurate in images
with a bright wide
region-like sky. VDSR is a well-known highly accurate SISR architecture
inspired by VGG-net used for ImageNet classification with 20 weight layers.
Long threaded cascading small filters form a deep network structure, and
correlated information in wide image regions is extracted efficiently. This
framework is faster than SISR and the learning convergence rate is also high at
training due to adjustable gradient clipping [4].
The area is of
great interest and authors
have been
carrying out research
in
this
field
with
prior works
[6-9, 30].
The dehazing problem
is predominantly ill-posed. By designing an efficient regularized filter,
dehazing problems can be resolved with the refinement of the coarse estimated
airlight.
Over-saturation, halo artifacts, and gradient reversal artifacts
problems
have been found in most existing dehazing techniques. Explicitly,
the proposed technique refines the TM via super resolution [4] based DM
estimation. The technique implements residual learning with significantly high
learning rates to optimize a very deep network fast. Convergence speed is high
with gradient clipping to ensure training stability. The effectiveness of the
algorithm is tested in terms of subjective and objective comparison with the
state-of-the-art techniques [17- 22, 29] on three benchmark datasets. This
comparison study shows sufficient improvements in DM, TM and following high
quality outputs.
The major contributions of the
proposed method are:
We propose a
novel single image VIA (SIVIA)
to minimize the effect of poor visibility.
The procedure
is
executed
in 3 steps. First,
low complexity SRVDSR based depth map estimation followed by TM estimation has
been proposed. The atmospheric scattering image formation model is employed to
get the original scene radiance. Finally, a super-resolution output image is
obtained from the inverting image formation optical model. Experimental results
are assessed and compared to the state-of-the-art VIAs using peak signal to
noise ratio (PSNR), structural similarity index (SSIM), Naturalness Image
Quality Evaluator (NIQE), Blind/ Referenceless Image Quality Evaluator
(BRISQUE), and CIEDE2000 showing improved performance. DMs are compared and
analyzed. The paper is organized as
follows:
a literature survey
presented in section 2, the proposed technique described in section 3, results
arranged in section 4, and section 5 containing the discussion.
Spotlight:
●
A
novel single image VIA (SIVIA) is proposed to minimize the effect of poor
visibility by
inverting
the atmospheric scattering image formation model with
super resolution.
●
Low
complexity SRVDSR based depth map estimation followed by TM estimation,
comparison and analysis, which results in noise reduction.
●
The
lower bound of transmission has been examined in detail.
●
Experimental
results are assessed and compared to seven state-of-the-art VIAs using the peak
signal-to-noise ratio (PSNR), structural similarity index (SSIM), Naturalness
Image Quality Evaluator (NIQE), Blind/ Referenceless Image Quality Evaluator
(BRISQUE), and CIEDE2000 show improved performance.
●
SIVDSR-Dhaze,
SIVDSR-Dhaze
RE outputs obtained.
Light waves encounter numerous microscopic
particles as they traverse the transmission medium to reach the image sensors.
Atmospheric
scattering
occurs
which results
in higher luminance.
However,
the reflected light waves are
themselves
also
attenuated and scattered along the path to the image sensors.
Such, multiplicative
loss causes a
decrease in image contrast [10]. Visible light is primarily attenuated in the
atmosphere by the scattering phenomenon
and
the process model is controlled with an
extinction
coefficient k. Fog is solely responsible for this phenomenon and creates a
luminous veil on the way of direct transmission hindering visibility [14]. In
1924, Koschmieder [13] developed a simple relationship between the apparent
luminance L of an object at a distance d and its intrinsic luminance L0
and Lf
as far end object luminance.
|
(1)
|
In 1975, McCartney proposed the optic based image formation model
which is frequently used in image processing applications. The detailed
description was presented by Narasimhan in 2000 [35, 36] which modelled
atmospheric scattering taking into
account the
resulting
attenuation. Many renowned methods have been developed from this model
such
as
the multiple images fusion method, partial differential equation algorithm,
Tan method, Fattal method, Markov Random Field with Bayesian algorithm, He
method, and so on. The image formation optical model as shown in figure 1, d
stands for the distance between objects of interest and the observer, β
(λ ) represents the atmospheric scattering coefficient, and λ denotes
the wavelength of the light [11, 26, 36].
Figure 1. Optical Image Formation Model [28]
In [2], atmospheric effects are removed from terrain images taken
by a forward-looking airborne camera assuming the airlight to be constant over
the entire image with known scene depth. Dark Channel Prior (DCP) [18] produces
good results with a patch-wise assumption of minimum intensity channel in RGB
image. However, some images do not produce satisfactory results due to an
underestimated transmittance over dehazing and colour distortion. DCP is a
statistical observation for TM estimation accurately. DCP is based on the dark
object [1] and the key observation is
mostly formed by
the local regions
(except the sky or hazy regions)
that
contain pixels
with
low intensity in at
least one of the colour channels. To reduce artifacts, the transmission
estimate is further refined based on the alpha matting strategy.
Meng
further explored the merits of the DCP [18]. The technique incorporates a
boundary constraint which is a weighted L1
norm regularization on
the transmission estimation produced by DCP. Thus, the overall drawback of DCP
alleviates, and sharp edges and bright sky regions are addressed efficiently
[19].
Fattal
proposed a technique depending on the inspection of the
distributions of pixels in a small patch of a natural image. These patches
stand for one-dimensional structures known as colour lines, in the RGB plane. A
low complexity TM is estimated from the computed
offset of
colour lines to the
origin. A refined transmission is formed by a Markov random field model
removing the noise and other artifacts due to scattering [20].
Berman et al.
further investigated the colour consistency observation that examines the
colour distribution in a haze-free image. This is well approximated by a
discrete set of clusters in the RGB colour space. Furthermore, these pixels in a
given cluster are non-local and distributed over the entire image plane. Thus,
these pixels are affected asymmetrically by the haze forming cluster leading to
haze line. The position of each pixel within the line estimates its
transmission level. These haze lines
highlight
the inconsistency of
transmission in different regions of the image. Finally, this is applied to
estimate the TM [21].
Cai et al.
recommend an end-to-end deep CNN model
training with synthetic haze to haze-free patch mapping. The technique breaks
down into four consecutive steps: features extraction, multi-scale mapping,
local extrema, and finally non-linear regression [29].
Ren et al.
proposed
a multi-scale CNN to estimate the TM directly from hazy images trained with
synthetic hazy images generated from haze-free images. In turn, the DM is
computed followed by application in the light propagation model. The TM is
first estimated with a coarse-scale network followed by a fine-scale network
[22].
Ancuti et al. present
a novel straightforward method for local
airlight estimation along with the advantage of the multi-scale fusion strategy
fusing the multiple versions obtained from distinct definitions of the locality
notion. The algorithm is equally effective for the complex night-time dehazing
challenge and day-time hazy scene improvement with severe scattering or
multiple sources of light [17].
LR images can
produce corresponding numerous HR images making SISR image a challenging
intractable problem [23]. Earlier mapping of HR images from LR images was
insufficient and inefficient. Recent deep learning (DL) based SISR methods are
efficient subjectively and objectively. SRCNN was a benchmark for network
architecture [24]. SRCNN consists of three-layer CNNs, each with the filter
sizes of each layer 64×1×9×9, 32×64×5×5 and
1×32×5×5. These three nonlinear transformations perform patch
extraction, nonlinear mapping and reconstruction. The loss function for optimization
is mean square error.
5-10 layers of feature learning are common for
convolution neural networks (CNN). 50-100 layers of CNNs form very Deep CNN.
Very Deep Super
Resolution (VDSR) improves resolution with a very deep learning technique [4].
With the increase in depth of DL architecture, it becomes difficult to train
CNN. Recently, techniques have been developed to converge faster with better
results [25]. Huang et al. [31,32] framework was used as a benchmark for
comparison of the state-of-the-art results taking into account the same
assessment process. Bicubic interpolation is adopted in the YCbCr colour model
to take into account the human visual sensitivity to the intensity rather than
colour information. Finally, the luminance component is enhanced and denoised
with the VDSR network via the final regression layer of the activation network.
CIEDE2000 colour distortion is an important parameter for distortion in colour
of an image after processing [33, 37].
In
this section we describe the proposed
single image very deep super
resolution SIVDSR [4] based dehazing technique.
We consider the
technique application
from the prospect of the image formation optical model [10]. The
VDSR network has been employed for better denoising with super resolution
module in the depth estimation stage. Finally, we employ contrast enhancement
on the radiance image formed by inverting the image formation optical model.
If N denotes the number of pixels in an RGB colour
image I, then 3N possible numbers of equations can be possible. The image
formation scattering model in figure 1 is represented as:
|
(2)
|
Here,
,
, and
are scalers with
the colour components of the channel
.
has 3N unknown
radiance. Whereas, there are N unknown transmission t, and 3 unknown
atmospheric light
. Thus, there are 4N
+ 3 numbers of unknowns compared to 3N number of equations as in figure 2. This
leads to an ill-posed problem that arises due to the ambiguity in the spatially
varying t, and hence generating N variables. Thus, at least one extra
constraint is required for each pixel to solve the ambiguity [5, 30].
Figure 2. The colour vector I
represents a linear combination of J and A in the RGB colour space [18].
Equation 2 can be rewritten as equation 3.
|
(3)
|
I(x) represents an
observed or received image captured by a camera or sensor.
J(x)
stands for hazefree
image or original scene radiance.
t(x)
is the transmission
map representing the part of the light that arrives the camera without
scattering.
Astands
for the global atmospheric light.
J(x)
t(x)
describes direct decay of light or attenuation due to the scattering of the
reflected light in the medium and is expressed as
direct attenuation.
A(1-t(x)) stands for the atmospheric optical shifted component and is named as
airlight.
This is the activity of the atmospheric light with its shift as a result of
previously scattering of light, and as a result original scene colour shifts.
In equation 3, J(x)t(x) is a multiplicative effect on the scene radiance,
whereas A(1-t(x)) is an additive effect. In nutshell, original scene radiance
is influenced by multiplicative attenuation phenomena followed by additive
airlight shifting which leads to colour fainting as well as shifting.
Transmission of the medium represents homogeneous atmosphere as
|
(4)
|
According to
Dark
Channel Prior (DCP) [18], a natural clear RGB image has at least one
minimum intensity patch in a channel out of three channels. This minimum
intensity patch is almost zero value. This has been observed in over 5000 image
datasets.
Now in [18], t(x) is
readdressed as
|
(5)
|
The corresponding
received image is as
|
(6)
|
|
(7)
|
JC
indicates scene radiance in a channel of RGB image. JDark
represents
dark channel of JC. In correspondence to JDark, observed
minimum intensity channel is Imin. Again, considering aerial
perspective, t(x) can be rewritten as
|
(8)
|
ω,
the haziness factor
adds a realistic appearance to the dehazed image. Thus, finally inverting the
image formation optical model of equation 3, the output dehazed image is as
|
(9)
|
where
t0
is the lower bound of the
transmission map
t(x) and is set as 0.1[18]. From equation
9, when transmission t(x) reaches zero, J(x) becomes an ill-posed equation. To
overcome the issue, t0
is introduced. In [18], A is estimated
as 0.1% in the dark channel. In the proposed scheme, atmospheric light consists
of 0.1% brightest pixels of each channel which is more realistic and
appropriate. Moreover, atmospheric light and DM are estimated in parallel,
reducing computational time.
As discussed above,
the DCP technique generates good quality dehazed images at the expense of
computational cost. To overcome this, fast depth map estimation is found in the
literature [6-9, 30].
The incorporated in dehazing the i
mage formation
optical model [10]
is a
fundamentally ill-posed inverse problem. Thus, DM estimation
cannot be obtained in any one way or technique, rather infinite possibilities
are there.
Whithin
this proposed dehazing
approach,
super resolution
technique VDSR [4, Matlab2018a] has been implemented. DM is estimated in a
minimum intensity channel denoised by VDSR.
DM estimates the amount of low intensity in the RGB image. The
main objective in DM is to find the amount of scene radiance removing noise
that corrupts the image during capturing or due to attenuation [6-9, 16-22].
The DM also provides the depth of distance in an image. In the primary stage,
DM is considered the minimum intensity channel out of three RGB channels.
Still, it has some noise. To refine DM, super resolution based residual image
technique [4] has been incorporated.
TM
indicates the amount
of light that reaches the camera without attenuation considering the atmosphere
to be homogeneous. This transmission is achieved by inverting or complimenting
the DMs. Thus, it is better to consider the residue from a unitary image of the
same size as the original image [18]. It is defined in equation 8.
3.1.2.1 Importance
of Lower Bound of Transmission Map (t0)
Transmission is kept
under control by a lower bound (t0) as haze prevails in dense haze
regions. Transmission may fall at a value of zero making equation 8 an
ill-posed inverse problem. To restrict this situation, the lower limit of TM is
mandatory. In [18], the lower limit is considered 0.1 as discussed in section
3.1. The variable lower limit of TM has not yet been considered.
As mentioned in section 3.1, equations 8 and 9, the atmosphere on
a clear day contains a certain amount of particles. These particles create haze
at distant objects. Furthermore, the aerial perspective of human vision is the
fundamental cue to estimate depth. Due to this effect, distant objects look
faint [18].
ω
is the parameter known as the haziness factor and most of the
SIVMs incorporate it as a constant. But in [7-9], this is the ratio of minimum
intensity to maximum intensity in a depth map. Thus,
ω
is an adaptable parameter
depending on the captured atmosphere [6-9] as shown in equation 8.
3.1.3.1 Adaptable Haziness
Factor
In [7-9, 30], the
adaptive haziness factor was introduced to address the ever changing and
inhomogeneous atmosphere producing effective results. It was already
established that one of the channels of RGB image is darker than the other two
channels, and one patch of that channel is darkest. With this assumption, the
adaptable haziness factor is conceptualized as the ratio of minimum intensity
pixel to maximum intensity pixel in the dark channel or DM channel. This
assumption has also been incorporated into these proposed techniques.
We have proposed a
dehazing model in this paper which is novel from the viewpoint/aspect of
obtaining the improved restoration of weather corrupted images. This model can
be divided into three parts: the atmospheric scattering model, followed by DM
estimation in the VDSR network, and enhancement with residual images.
A DM of
low visibility images has been refined through a minimum order statistics
filter of linear complexity O(n) followed by improved transmission estimation.
An adaptable haziness factor has been proposed by a ratio of minimum intensity
to the maximum intensity of the 3-D RGB image vector [7-9]. The extinction
coefficient of atmosphere and visible distance (the distance that can be viewed
in the image) before and after processing have been presented for each image
automatically. Moreover, a simple contrast metric has been applied as the
difference between maximum intensity to minimum intensity of 3D RGB image
vectors. By inverting the image formation optical model, a clean image is
produced
by application of
the model in section one. The model
is applicable for diverse types of low visibility images. The MOSF used for TM
recovery is unique in this research work and has been efficiently applied in
the algorithms as the transmission estimator which is the main motivation for
the fast and good performance of the proposed methods. Dark channel prior is a
patch-based technique that gives patch information, whereas MOSF gives
artifact-free detailed pixel information at the expense of minimum cost. Depth
map and atmospheric light estimation are very important in colour image
dehazing. Selecting the correct atmospheric light value is a challenging
problem. In our work, the new value is obtained by first estimating the top
0.1% of the brightest pixel in the minimum channel, which mostly is the haze
opaque part of the image and the brightest portion of the image followed by the
value of the bright channel. Then, the average of the two channels is
collected. This method overcomes the deficiency of the dark channel prior and
reduces the influence of white objects or sky areas on the whole image. Moreover,
scene radiance dehazed image is improved with the VDSR network as shown in
figure 3 as the block diagram of the proposed technique. A revised model of
figure 3 is shown in figure 4. In figure 4, the depth map is refined with VDSR
[4] which develops good quality TM following high quality radiance image.
Figure 3. Block
Diagram [40]
Figure 4. Block
diagram of the proposed SRVDSR-Dehaze model, A represents atmospheric light, DM
as a depth map, TM as a transmission map
Table I: SIVDSR
Algorithms [40].
Algorithm: I (SIVDSR)
|
1:
|
Input hazy image
|
2:
|
Atmospheric light estimation,
|
3:
|
Depth map estimation
|
3:
|
Transmission map estimation
|
4:
|
Scene radiance recovery by Image formation Optical model
|
5:
|
Output Dehazed Image with VDSR
|
Table II:
SIVDSR-Dehaze: The Proposed Model I.
Algorithm-II (Modification of algorithm I)
|
1:
|
Input hazy image
|
2:
|
Atmospheric light estimation,
|
Refined Depth map estimation through super resolution technique
(three stages: raw depth map (minimum of three channels), 2D bicubic super
resolution image, 2D residual image) VDSR network [4]
|
3:
|
Refined Transmission map estimation
|
VDSR RGB Image
|
4:
|
Refined Scene radiance through Image formation Optical model
|
In algorithm-II, step-II consists of two parallel
module-atmospheric light estimation and depth map estimation. Due to parallel
operation, step II saves time. This is an added advantage. Algorithm III is by
product of algorithm II where the output of algorithm II is enhanced with VDSR
RGB, refined RGB image reference with the hazy image as in equation 13.
Table III:
SIVDSR-DehazeRE Dehazing: The Proposed Model II.
Algorithm-III (Modification of Algorithm II)
|
1:
|
Input hazy image
|
2:
|
Atmospheric light estimation,
|
Refined Depth map estimation through super resolution technique (three
stages: raw depth map (minimum of three channels), 2D bicubic super
resolution image, 2D residual image) VDSR network [4]
|
3:
|
Refined Transmission map estimation
|
VDSR RGB Image
|
4:
|
Refined Scene radiance Image formation Optical model
|
5:
|
Output super resolution residual enhanced Dehazed Image
|
VDSR [4] is the
first pioneering work with Single Image Super Resolution (SISR). As shown in
figure. 6, VDSR is a 20-layer VGG-net with a 3x3 kernel. Initially, the
learning rate is high for fast convergence. Gradient clipping is used to reduce
gradient explosion artifacts. Apart from novel architecture, VDSR has
established two more attributes, i) a single model for multiple scales as the
SISR operates with different scale factors having a strong relationship with
each other. This concept inspires many classical SISR methods. As compared with
Super-Resolution Convolutional Neural Network (SRCNN), bicubic interpolation on
Low Resolution (LR) image acts as input to the VDSR network. In the training
phase, LR bicubic input images of different scale factors are processed in
VDSR. The mapping with a smaller scale factor (×2) may also be
instrumental for a higher scale factor (x3 or x4), ii) the residual learning is
the other unique attribute. Mapping from the bicubic version to High Resolution
(HR) is not direct, rather VDSR employs deep CNN for learning the mapping from
the bicubic to the residual between the bicubic and HR. The residual learning
improves performance and accelerates convergence
(see
figure 7).
Figure 5. Schematic
of the Super-Resolution [4, Matlab18a]
Figure 6. VDSR
Network [4]
Hyperparameters used
and their attributes:
The parameters used to train
our final model are as follows. A network of depth 20 with batch size 64 is
considered for training. The momentum and the weight decay parameters are set
at 0.9 and 0.0001, respectively. Weights are initialized as [12] with rectified
linear units (ReLU), which is a robust and effective module in the network. The
network is trained with all images amidst 80 epochs (9960 iterations of 64
batch size). The learning rate is initially set at 0.1 following a decrease by
a factor of 10 in every 20 epochs. In short, the learning rate decreased 3 times
with learning stopped after 80 epochs and training time almost 4 hours on GPU
Titan Z.
Residual Learning
Architecture:
Figure 7. L-R:
Traditional CNN framework, Residual Framework
The traditional CNN
framework and Residual framework are shown in figure 7. There is a shortcut in
the residual framework between every two consecutive layers. This is called a
skip
connector. One such block is called the
Residual network
as in
figure 7. Dimension mismatch frequently occurs due to convolution operations in
those two consecutive layers. This problem is alleviated by identity mapping
for X and F(X). After resizing F(X), H(X) is generated with F(X) and X of the
same size. F(X) is the predicted residual,
|
(10)
|
Thus, F(X) and X are
of the same size. Finally, residual Image is predicted as
|
(11)
|
As r is considered
as an actual residual image, so that
|
(12)
|
The loss function is
as low as possible.
Loss
layers collect three inputs; residual estimate, network input (ILR image) and
ground truth HR image. The loss is determined as the Euclidean distance between
the reconstructed image (the sum of network input and output) and the ground
truth.
This is the last
module of SIVDSR techniques as in figure 4, and Table III. The difference
between the SR image and the hazy image is added to the dehazed output for
enhancement. In
Residual Enhancement step V,
the difference between the
VDSR image from step III and the hazy image is amplified and added with the
refined scene radiance from step IV. The result is shown in figure 8 below with
scene 41, the O-Haze dataset. A comparative visual representation is shown in
figure 9, with the left to right images as Hazy, VDSR, Residual, Dehaze, and
Residual Enhancement.
|
(13)
|
Figure 8. Residual
Enhancement
Figure 9. Image L-R:
Hazy, VDSR, Residual, Dehaze, Residual Enhancement
The effect of the lower bound of transmission t0
(0.9,
0.5, 0.1) on the dehazed image is shown in figures 9, 10, and 11. In [18], the
lower bound of transmission has been fixed at 0.1. In this proposed work,
different values of transmission lower bound have been incorporated as in eq.
(9) and investigated its effect in figures 10, 11, and 12; and tabulated in the
corresponding tables IV, V, and VI. Two types of dehazed images have been
examined, super-resolution dehazed image and enhanced super-resolution dehazed
image. In all three cases of lower bounds, significant results have been found
and listed in Tables IV, V, and VI. Parameters (PSNR, SSIM, NIQE, BRISQUE) are
used for evaluation and effective results are found at 0.5, and 0.9
respectively.
Figure 10. t0=0.9,
L-R: Hazy Image, Super Resolution Image, Super Resolution enhanced Image
Table IV: Performance with
figure 10
t0=0.9, ω=0.6
|
|
Hazy Image
|
SR radiance image
|
Enhanced SR
radiance image
|
Full reference
parameter [14, 13]
|
PSNR
|
-
|
53.0888
|
52.6401
|
|
SSIM
|
-
|
0.9798
|
0.979
|
No reference
parameter [16, 15]
|
NIQE
|
1.9013
|
2.5082
|
3.0171
|
|
BRISQUE
|
25.1514
|
17.449
|
34.8778
|
Figure 11. t0
=0.5 L-R: Hazy Image, Super Resolution
Image, Super Resolution enhanced Image
Table V: Performance of figure
11.
t0 =0.5, ω=0.6
|
|
Hazy Image
|
SR radiance image
|
Enhanced SR radiance image
|
Full reference parameter [14,
13]
|
PSNR
|
-
|
52.1919
|
52.0621
|
|
SSIM
|
-
|
0.9717
|
0.9738
|
No reference parameter [16,
15]
|
NIQE
|
1.9013
|
2.9567
|
5.927
|
|
BRISQUE
|
25.1514
|
19.2671
|
36.3221
|
Figure 12. t0
=.1 L-R: Hazy Image, Super Resolution Image, Super Resolution enhanced Image
Table VI: Performance of figure
12.
t0 =0.1, ω=0.6
|
|
Hazy Image
|
SR radiance image
|
Enhanced SR
radiance image
|
Full reference
parameter [14, 13]
|
PSNR
|
-
|
52.1732
|
52.0057
|
|
SSIM
|
-
|
0.9716
|
0.9735
|
No reference
parameter [16, 15]
|
NIQE
|
1.9013
|
2.9615
|
6.1707
|
|
BRISQUE
|
25.1514
|
19.1983
|
35.5928
|
In this section, the proposed SIVDSR-Dehaze net has been described
in detail with a histogram and scatter plot. As shown in figures 13, 14, and
15, SIVDSR-Dehaze net performance has been evaluated considering the canon.png image
[18] as a reference. Two techniques are used. The second technique is the
enhancement version of the first one as described in sections 3.2.1 algorithm
II, and III. In figure 13, histograms show a clear effect of haze and haze free
images of the same scene. The histogram of hazy images reflects dense pixel
orientation towards the centre of the graph; whereas histograms of the haze
free images are well distributed and Gaussian in nature representing good
contrast. Figure 14 illustrates the scatter plots of a) hazy image, b) SR
image, c) EnhancedSR image, and d) GT. These scatter plots enumerate the
richness in colour and contrast of the proposed techniques in comparison with
the hazy ones. The hazy -
dehazy image colour cloud plots and histogram are interesting and
important in understanding the effectiveness of dehazing schemes. It has been
illustrated in figures 13 and 14 by the histogram and colour cloud plotting of
the 13_outdoor_hazy.png image under the O-Haze data set [17]. It is clearly
evident that in hazy images pixel intensities are clustered densely around the
upper central part of the intensity scale [0-255]; whereas, in the case of
hazefree images, pixel intensities are loosely distributed in almost the entire
intensity scale. DM, TM, and radiance images of the same canon.png image have
been shown in figure 15. It is evident that DMs are darker in the case of haze
free images. Consequently, haze free TMs reflect more visibility.
Figure 13. Histogram, L-R: Hazy image, Super resolution image,
Enhanced Super resolution image, and GT.
Figure 14. Colour cloud plot of L-R: Hazy image, super resolution
image, Enhanced SR image, and GT.
Figure 15. Row-1,
L-R: Image: Haze, Super Resolution, Enhanced Super Resolution, GT
Row-2, L-R: Depth
Map: Haze, Super Resolution, Enhanced Super Resolution, GT
Row-3, L-R: Transmission Map: Haze, Super Resolution, Enhanced
Super Resolution, GT
Hazy DM, residual
DM, SIVDSR-Dehaze DM are shown in figure 16. Corresponding TMs and radiance
images are shown in figures 17, and 18 respectively. In figure 18, effective
parametric evaluation results are also shown.
Figure 16. Depth
map, L-R: Residual Depth Map, Raw depth map, Super resolution Depth Map
Figure 17.
Transmission map of figure 16,
L-R: Residual TM, Raw TM, Super resolution TM
Figure 18. Scene Radiance of scene 13[17], Dehazed output, L-R:
Residual Dehazed output, Raw Dehazed output, Super resolution Dehazed output,
self-adjusting haziness factor= 0.6423 [7-9], PSNR= 53.5857, 52.1978, 51.4765,
SSIM= 0.9819, 0.9715, 0.9635.
In sections 3.1.3
and 3.1.3.1, aerial perspective and adaptable haziness factor were discussed.
Here, their effects are investigated. In figures 19 and 20, fixed haziness
factors of 0.9 and 0.65 are implemented with hazy, SIVDSR, Residual,
SIVDSR-Dhaze, and SIVDSR-DhazeRE images. Furthermore, parametric evaluation
(PSNR, SSIM) shows improved results along with qualitative appearances.
The appearance of
the hazy image with respect to ground truth, super resolution dehaze, and
enhanced super resolution dehaze images are shown in figure 19 along with the
depth map and transmission map. The proposed results are visibly improved with
increased contrast and higher resolution than GT. Moreover, halo,
oversaturated, and gradient reversal artifacts are removed.
Figure 19. Fixed
haziness factor (0.9), L-R: hazy, SIVDSR, Residual, SIVDSR-Dehaze,
SIVDSR-DhazeRE, GT
Row-1:
43_outdoor_hazy.png: PSNR, 15.7524, 13.5508; SSIM, 0.9979, 0.9966
(SIVDSR-Dehaze, SIVDSR-DhazeRE respectively);
Row-245_outdoor_hazyPSNR,
12.8084, 12.1036; SSIM, 0.9799, 0.9843 (SIVDSR-Dehaze, SIVDSR-DhazeRE respectively).
Figure 20. Adaptable
haziness factor 0.65, L-R: hazy, SIVDSR, Residual, SIVDSR-Dehaze,
SIVDSR-DhazeRE, GT
Row-1:
43_outdoor_hazy.png, PSNR,16.9375, 14.2856; SSIM, 9987, 9965 (SIVDSR-Dehaze,
SIVDSR-DhazeRE respectively);
Row-2: 45_outdoor_hazyPSNR,
13.8708, 12.9066; SSIM, 0.9814, 0.9862 (SIVDSR-Dehaze, SIVDSR-DhazeRE
respectively);
In this section,
canon.png [18] with dense haze has been tested with our methods using different
lower bound of transmission. In figure 21, at t0
0.1 SIVDSR-DhazeER
produces oversaturated, halo, and gradient reversal artifacts. With the
increase of t0, these artifacts are not prominent as in figures 22,
and 23.
Figure 21. at t0=0.1, L-R: Hazy Image, Oversaturated,
gradient reversal, halo effect outputs with the proposed methods
(SIVDSR-Dehaze, SIVDSR-DehazeRE)
Figure 22. t0
= 0.5, L-R: Hazy Image, Oversaturated, gradient reversal, halo artifacts
outputs with the proposed methods
Figure 23. t0
= 0.9, L-R: Hazy Image, Oversaturated,
gradient reversal, halo effect free outputs with the proposed methods
The lower bound of transmission has been experimented with in
figures 21, 22, and 23 as 0.1, 0.5, and 0.9 respectively with SIVDSR-Dehazed and
SIVDSR-DehazeRE outputs. The image is canon.png of size 500x500 [18]. Figures
21, and 22 are showing oversaturated, gradient reversal and halo artifacts
resulting at t0
0.1, and 0.5 respectively. In figure 20 with t0
as 0.9, clear outputs are shown. Thus, this can be established that the
lower bound of transmission plays an important role in producing clear outputs.
Finally, the fixed value of t0
may not produce the desired output.
This leads to variable or adaptable lower bounds of the transmission map estimation.
In figures 24, 25, and 26, a detailed study of DM, TM at t0
0.1.
0.5, 0.9 have been observed.
Figure 24. Hazy
Image, Oversaturated, gradient reversal, halo effect outputs with the proposed
method at t0=0.1,
Row-1: Haze,
SIVDSR-Dehaze, SIVDSR-DehazeRE; Row-2:DM of row-1; Row-3:TM of row-1.
Figure 25. Hazy
Image, Oversaturated, gradient reversal, halo effect outputs with the proposed
method at t0=0.5
Row-1: Haze,
SIVDSR-Dehaze, SIVDSR-DehazeRE; Row-2:DM of row-1; Row-3:TM of row-1.
Figure 26. Hazy
Image, Oversaturated, gradient reversal, halo effect free outputs with the
proposed method at t0=0.9
Row-1: Haze,
SIVDSR-Dehaze, SIVDSR-DehazeRE; Row-2:DM of row-1; Row-3:TM of row-1.
The proposed method is implemented on MATLAB R2018a on a PC with a
2.8 GHz Intel Core 2 Duo Processor. Our proposed model experimented with the
O-Haze dataset [17], and [18] to conduct a comprehensive evaluation of the
state-of-the-art single image dehazing techniques presented.
Benchmark Used
Benchmark Dataset is
used [17-22].
The details of the generator structures and parameter settings are
shown in Table VII.
Table VII. Parameter for performance evaluation
Sl. No.
|
Parameter
|
Requirement
|
Type
|
1
|
Peak Signal to
Noise Ratio (PSNR) [14]
|
High value
|
Full Reference
|
2
|
Structure
Similarity Index Metric (SSIM) [13]
|
[0-1] in
normalized scale high value
|
Full Reference
|
3
|
blind/reference
less image spatial quality evaluator (BRISQUE) [15]
|
Smaller value
better performance
|
No reference
|
4
|
Naturalness
Image Quality Evaluator
(NIQE) [16]
|
Smaller value
better performance
|
No reference
|
5
|
CIEDE2000[33,
37]
|
Lower value for
low colour distortion
|
No reference
|
Table VIII. Quantitative Evaluation of figure 27 (PSNR, SSIM) with
image size 350x350x3
Parameter
|
Dehazed Image
|
Improved Depth Map
|
Improved TM
|
PSNR
|
12.0783
|
12.4972
|
20.244
|
SSIM
|
0.4374
|
0.4949
|
0.6474
|
Table IX. Quantitative Evaluation of figure 27 (BRISQUE, NIQE)
Parameter
|
Hazy Image
|
Depth Map
|
TM
|
Improved Depth Map
|
Improved TM
|
Dehazed Image
|
BRISQUE
|
18.8784
|
30.5624
|
12.9649
|
43.4582
|
43.4582
|
18.8747
|
NIQUE
|
24.4850
|
18.8814
|
18.8793
|
18.8772
|
18.8768
|
43.4582
|
Figure 27. Qualitative Evaluation of gugon.png [18],
L-R: Top row- (a)
degraded image, (b) residual image, (c) VDSR image, (d) clean image
Bottom row- (e) Hazy
DM, (f) TM, (g) recovered DM, (h) recovered TM.
Figure 27
illustrates one scene (gugoon.png) [18] as input with our model and each step
output has been extracted and compared qualitatively and quantitatively w.r.t
hazy counterpart as shown in tables IX, and X.
In figure 28, one
hazy image-GT pair [17] is tested with the state-of-the-art methods [17-22,
29], and our methods. Its parametric evaluation is shown in Table X. Figure 29
is the crop/ zoom version of figure 28 for a detailed study.
In order to show the
efficiency of the proposed method, we compare results with related traditional
methods. This approach
performs
appreciably with a very dense
haze. However, results are often over-saturated because the method does not
utilize a physical model to recover the image. The results of our method show
clearer images with less colour saturation. The peak signal-to-noise ratio
(PSNR) is most commonly used to measure the quality of reconstruction of hazy
images. PSNR is most easily defined via the mean squared error. In our
experiment, we use PSNR, SSIM, NIQE, BRISQUE, and Entropy work in [4]. Table XI
shows comparable satisfactory results with the proposed methods.
Figure 28. Subjective evaluation with O-Haze dataset [17]. (a)
Hazy image, (b) Ground Truth, (c) SIVDSR-Dehaze, (d) SIVDSR-DehazeRE, (e)
DCP [18], (f) Meng [19], (g) Fattal [20], h) Berman [21], (i) DehazeNet [29],
(j) Ren [22], (k) Ancuti [17]
Table X. Evaluation of Algorithm Performance w.r.t figure 28
(PSNR, SSIM, NIQE, BRISQUE, Entropy) along with ranks
|
PSNR
|
SSIM
|
NIQE
|
BRISQUE
|
Entropy
|
GT
|
16.7986(2)
|
0.6818(5)
|
4.1515(9)
|
20.1243(10)
|
7.5171(6)
|
SIVDSR_Dehaze
|
16.581(3)
|
0.6482(6)
|
4.71(10)
|
4.4171(3)
|
7.0958(2)
|
Dehazed
|
13.3872(6)
|
0.6475(7)
|
3.6772(5)
|
10.4409(5)
|
7.7466(10)
|
DCP
|
11.6003(10)
|
0.5949(9)
|
3.5251(2)
|
2.5369(1)
|
7.6295(9)
|
Meng
|
11.9278(8)
|
0.7142(4)
|
3.532(3)
|
19.2958(9)
|
7.0896(1)
|
Fattal
|
11.9241(9)
|
0.6217(8)
|
3.8544(8)
|
11.4575(7)
|
7.211(3)
|
Berman
|
13.0319(7)
|
0.7232(3)
|
3.6336(4)
|
13.6197(8)
|
7.333(5)
|
DehazeNet
|
14.9516(4)
|
0.8383(2)
|
3.6809(6)
|
10.5405(6)
|
7.5696(8)
|
Ren
|
17.161(1)
|
0.9019(1)
|
3.4389(1)
|
3.6508(2)
|
7.2983(4)
|
Ancuti
|
13.8383(5)
|
0.3273(10)
|
3.7013(7)
|
8.2222(4)
|
7.5529(7)
|
Figure 29. Crop version of figure 27 [17]
The O-Haze dataset [17] has been used for comprehensive
performance evaluation on recent single image dehazing. Few non-homogeneous
scenes have been picked randomly from [17]. In figure 30, subjective results
are: L-R: the hazy image, He et al. [18], Meng et al. [19], Fattal [20], Cai et
al. [29], Ancuti et al. [17], Berman et al. [21] and Ren et al. [22], GT,
SIVDSR, and SIVDSRRE. T-B: 11 scene from 45 scene of O-Haze dataset [17]. Table
XI presents Quantitative evaluation of figure 30 and computes the SSIM and
CIEDE2000 indicating between the ground truth images and the dehazed images
produced by the evaluated techniques as mentioned above. Table XII Shows
quantitative evaluation of all the 45 set of images of the O-HAZE dataset. This
table presents the average values of the SSIM, PSNR and CIEDE2000 indexes, over
the entire dataset. Finally, figure 31 elaborates Qualitative Comparative
detail insets (cropped version) results as: L-R: the hazy image, He et al.
[18], Meng et al. [19], Fattal [20], Cai et al. [29], Ancuti et al. [17],
Berman et al. [21] and Ren et al. [22], GT, SIVDSR-Dehaze, and SIVDSR-DehazeRE.
T-B: 3 scene scenes (10, 19, 41) from 45 scene of the O-Haze dataset [17].
Figure 30.
Qualitative
Comparative results: L-R:
the hazy image, He et
al. [18], Meng et al. [19], Fattal [20], Cai et al. [29], Ancuti et al. [17],
Berman et al. [21] and Ren et al. [22], GT, SIVDSR-Dehaze, and SIVDSR-DehazeRE.
T-B: 11 scene from 45 sceneof O-Haze dataset [17].
Table XI.
Quantitative
evaluation.
We randomly picked up 11 sets from our O-HAZE dataset,
and
computed
the SSIM and CIEDE2000 indices
between the ground truth images and the dehazed images produced by the
evaluated techniques. The hazy images, ground truth and the results are shown
in figure 30.
|
He et al. [18]
|
Meng et al.
|
Fattal
|
Cai et al.
|
Ancuti et al..
|
Berman et al.
|
Ren et al.
|
SIVDSR-Dehaze(Ours)
|
SIVDSR-DehazeRE
(Ours)
|
SSIM[13]
|
CIEDE2000[33,34]
|
SSIM
|
CIEDE2000
|
SSIM
|
CIEDE2000
|
SSIM
|
CIEDE2000
|
SSIM
|
CIEDE2000
|
SSIM
|
CIEDE2000
|
SSIM
|
CIEDE2000
|
SSIM
|
CIEDE2000
|
SSIM
|
CIEDE2000
|
Set 1
|
0.82
|
22.37
|
0.77
|
21.06
|
0.73
|
24.29
|
0.58
|
24.42
|
0.75
|
20.09
|
0.76
|
20.97
|
0.81
|
18.17
|
0.9783
|
13.0719
|
0.9823
|
16.963
|
Set 6
|
0.74
|
19
|
0.78
|
11.44
|
0.73
|
21.89
|
0.59
|
16.16
|
0.68
|
15.53
|
0.77
|
12.68
|
0.72
|
13.2
|
0.9807
|
11.254
|
0.9812
|
13.2884
|
Set 10
|
0.78
|
15.22
|
0.76
|
16.63
|
0.75
|
17.49
|
0.71
|
16.17
|
0.73
|
19.21
|
0.72
|
17.77
|
0.8
|
13.7
|
0.9897
|
18.8753
|
0.9898
|
19.371
|
Set 19
|
0.81
|
16.31
|
0.84
|
13.37
|
0.79
|
21.48
|
0.72
|
16.92
|
0.78
|
15.55
|
0.82
|
14.49
|
0.83
|
12.98
|
0.9828
|
16.6627
|
0.9834
|
19.7795
|
Set 20
|
0.61
|
23.81
|
0.72
|
20.91
|
0.62
|
20.73
|
0.5
|
23.71
|
0.78
|
12.67
|
0.72
|
19.4
|
0.63
|
20.98
|
0.9886
|
13.8829
|
0.9857
|
15.7584
|
Set 21
|
0.69
|
27.5
|
0.78
|
21.13
|
0.63
|
28.25
|
0.71
|
19.49
|
0.78
|
10.72
|
0.72
|
20.54
|
0.73
|
20.26
|
0.9812
|
10.7899
|
0.9869
|
12.1366
|
Set 27
|
0.61
|
21.38
|
0.68
|
18.76
|
0.67
|
22.37
|
0.64
|
17.16
|
0.77
|
10.94
|
0.7
|
18.41
|
0.71
|
14.16
|
0.9861
|
9.498
|
0.9849
|
11.9723
|
Set 30
|
0.75
|
18.85
|
0.74
|
18.59
|
0.72
|
18.46
|
0.77
|
12.7
|
0.83
|
11.25
|
0.81
|
14.55
|
0.82
|
12.66
|
0.9829
|
12.8731
|
0.9831
|
13.3614
|
Set 33
|
0.76
|
18.54
|
0.74
|
15.84
|
0.76
|
17.86
|
0.81
|
14.61
|
0.61
|
20.86
|
0.66
|
19.39
|
0.88
|
10.87
|
0.9851
|
15.1021
|
0.9851
|
15.5605
|
Set 41
|
0.77
|
19.54
|
0.72
|
21.45
|
0.66
|
23.71
|
0.84
|
12.78
|
0.84
|
13.02
|
0.82
|
14.36
|
0.88
|
12.34
|
0.9853
|
11.1101
|
0.985
|
11.6285
|
Set 42
|
0.79
|
19.7
|
0.82
|
11.03
|
0.73
|
13.21
|
0.58
|
15.58
|
0.74
|
15.37
|
0.82
|
11
|
0.72
|
12.87
|
0.9841
|
12.126
|
0.9842
|
13.1674
|
Table XII. Quantitative evaluation of all the 45
set of images of the O-HAZE dataset. This table presents the average values of
the SSIM, PSNR and CIEDE2000 indexes, over the entire dataset.
|
He et al.
|
Meng et al.
|
Fattal
|
Cai et al.
|
Ancuti et al.
|
Berman et al.
|
Ren et al.
|
SRVDSR-Dehaze
|
SRVDSR-DehazeRE
|
SSIM
|
0.735(7)
|
0.753(4)
|
0.707(8)
|
0.666(9)
|
0.7470(6)
|
0.750(5)
|
0.765(3)
|
0.9840(2)
|
0.9846(1)
|
PSNR
|
16.586(5)
|
17.443(2)
|
15.639(7)
|
16.207(6)
|
16.855(3)
|
16.610(4)
|
19.068(1)
|
15.23(8)
|
14.13(9)
|
CIEDE2000
|
20.745(9)
|
16.968(5)
|
19.854(8)
|
17.348(7)
|
16.431(4)
|
17.088(6)
|
14.670(2)
|
13.204(1)
|
14.817(3)
|
Total Ranking
|
21
|
11
|
23
|
22
|
13
|
15
|
6
|
11
|
13
|
Ranking
|
5
|
2
|
7
|
6
|
3
|
4
|
1
|
2
|
3
|
Figure 31.
Qualitative
Comparative detail insets (cropped version) results: L-R:
the
hazy image, He et al. [18], Meng et al. [19], Fattal [20], Cai et al. [29],
Ancuti et al. [17], Berman et al. [21] and Ren et al. [22], GT, SIVDSR-Dehaze,
and SIVDSR-DehazeRE. T-B: 3 scene scenes (10, 19, 41) from 45 scene of the
O-Haze dataset [17].
The proposed algorithm is divided into two parts: i) Atmospheric
scattering model [2, 3, 10, 11], ii) VDSR NETWORK [4-5] as in figure 2. The low
complexity atmospheric scattering model has computational complexity O(n2)
[6-9, 30]. The VDSR model is faster than other SRCNN methods [4-5]. Thus, it
can be concluded that the proposed model works faster in the VDSR framework in
comparison with other SRCNN frameworks.
Applications:
a) Image visibility
improvement and Edge-preserving smoothing:
Here edge-preserving smoothing
filters can preserve the key features of an image like an edge and denoise the
image. Flash/No-Flash Denoising: It can denoise a no-flash image under the
guidance of its flash version.
b) Matting/Guided
Feathering:
Extracting foreground objects from an image means separating the
foreground from the background. It is used in video editing and image
processing.
c) Haze Removal:
Hazy images are
formed due to light scattering with particles in the atmosphere. Haze removal
filters will improve the image.
d) Joint Upsampling:
Under the guidance
of another image, upsampling is done. One application of joint upsampling is
the colourization of images.
e) Underwater
visibility improvement
A novel image dehazing
technique has been presented with a comparative evaluation with the seven
state-of-the-art contemporary techniques. In Section 4.3 scenes from the O-HAZE
dataset [17] are used in the experiments picked randomly. In figure 4, eleven
columns are there; the first column shows hazy images from [17], the second to
eight columns are [18,-22, 29, 33], column nine shows ground truth, and columns
ten and eleven are the proposed super resolution based dehazing results and
followed by enhanced super resolution based dehazing. Furthermore, in figure
31, the comparative cropped detail of scenes 10, 19 and 41, respectively have
been examined. In subjective observation, the work of He et al. [18] attains
73.5% of structure information SSIM [13] with low airlight estimation leading
to colour shifting. Additionally, low contrast and gloomy images are found.
Halo artifacts and gradient reversal effects are prominent. Thus, DCP fails in
the O-Haze dataset. In [19], Meng et al. proposed a method based on DCP
producing results better than [18] in low noise (PSNR), a low colour difference
(CIEDE2000) [33, 34] with good transmission estimation. Undesirable shifting of
colour is also prominent in Fattal [20], and in Berman et al. [21], the effect
is lessened with sharp edges with the local estimation of airlight and
transmission. In Ancuti et al. [17], the resulting images are of good contrast
due to efficient multi-scale fusion and local airlight estimation. Ren et al.
[22] (column 8 of figure3) and Cai et al. [29] (column 5 of figure 3) depend on
learning based techniques. Whereas, in [22] good results are obtained compared
to [29].
Finally,
the proposed approaches (SR Dehaze, tenth
column; and Enhanced SR Dehaze, eleventh column) present clear visible output
in both foreground and background. Thus, subjective analysis has been carried
out with the seven state-of-the-art techniques [17-22, 29]. They belong to
prior based techniques [18-20, 22, 17], and learning based techniques [21, 29].
Most of them, on average, produce meaningful results with halo artifacts,
gradient reversals, and colour shifting. Unpleasant, synthetic appearances are
inevitable.
It should be noted that,
with the O-Haze dataset [17]
ground truths (GT) are also available,
this
gives the
comparative study a robust validation. But proposed techniques present far
better output in comparison to all others. The effectiveness of the proposed
techniques is measured with the O-Haze dataset as the objective evaluation with
other dehazing techniques. In Table XI, a comparative analysis with the
different dehazing techniques prior based [18-21, 29, 27] and learned based
techniques [21, 22] with respect to GT and the proposed techniques have been
discussed against parameter SSIM, CIEDE2000 [13, 33, 34] as in figure 30. Local
patterns of pixel intensities of two images of the same scene are compared in
the structural similarity index (SSIM) normalizing luminance and contrast in
the range [0-1]; 1 for an absolute similar image, zero indicates no match at
all. maximum value 1 for two identical images. CIEDE2000 is a parameter for
colour distortion measurement in the range [0-100]; a smaller value for better
preservation of colour, and a higher value for high colour distortion.
The O-Haze dataset has 45
scenes, and an average of SSIM, PSNR, CIEDE2000 of the entire dataset scenes
has been entered in table XII experimenting with those seven techniques [17-22,
29] and two proposed techniques. Table XII shows our methods outperform other
methods in SSIM, and in CIEDE2000. In the case of PSNR, Ren et al [22] give the
best performance. The bottom row of Table IX shows that Meng et al [19]
outperform other methods (a total of eleven methods) in parametric evaluation
(SSIM, PSNR, CIEDE2000). The second and third positions are obtained by the
proposed methods. Meng et al [19] obtained 2nd position jointly and Ancuti et
al [17] secured 3rd position jointly. The following positions are [18, 20, 21,
29]. In nutshell, none of the methods executes better than the others with all
45 images in the O-Haze dataset. The highest value of SSIM and the lowest value
for CIEDE2000 are recorded with the proposed methods.
To summarise,
the above analysis
shows the complexity of the single-image based dehazing problem. Moreover, this
also has been established that the proposed methods produce convincing results
qualitatively and quantitatively. Finally, cropped version scenes of 10, 19, 41
show comparable results, even better.
In this article, a single image
super resolution based image visibility improvement method has been proposed,
which is designed for improving the visibility of digital images captured in turbid
atmospheres. Researchers are working to reduce poor visibility and SIVIAs are
most
prominent
but
also have
challenges of low computational complexity, accurate depth map,
and preserving the image quality and structure. The image formation scattering
linear model [10] is inverted with a significant change incorporating in-depth
map estimation leading to better transmission followed by a clear image. This
new method has been compared qualitatively and quantitatively against eight
state-of-the-art techniques; VDSR is employed to preserve important structures
of the image in depth map estimation. VDSR is a very deep network based on
super-resolution. In VDSR, residual learning with extremely high learning rates
is adapted to reduce the slow convergence rate with maximizing convergence
speed, gradient clipping is used to nail down training stability. VDSR is
applicable in image super-resolution, reconstruction, denoising, and
compression artifact removal. Due to these properties of VDSR, the depth map is
denoised effectively for reducing poor visibility. Enhanced SR Dehazing is a
by-product of the above mentioned algorithm. Thus, as a whole two algorithms
have been proposed. PSNR, SSIM, BRISQUE, NIQE, and CIEDE2000 are some important
parameters to evaluate image quality in VIAs. It is clear from the acquired
results that the proposed algorithms are superior in reducing the effect of
poor visibility.
Future Direction
-In the future, as shown in table XII
there is scope to improve parameter PSNR values so that the ranking of the
proposed algorithms will be increased.
Table XIII. Author’s Statement
Authors’ contributions
|
Single image super Resolution
dehazing image output is obtained with the VDSR network. Both the authors
have contributed equally to this work.
|
Ethical Conflict
|
There is no conflict of
interest in this work.
|
Funding
|
No funding has been received
for the project.
|
Data Availability Statement
|
NA
|
Table XIV.
Abbreviation used
VIA
|
SIVDSR
|
SIVDSR-Dehaze
|
SIVDSR-DehazeRE
|
TM
|
CNN
|
GT
|
Visibility Improvement
Algorithms
|
Single Image Visibility Super
Resolution
|
Single Image Visibility Super
Resolution Dehaze
|
SIVDSR-Dehaze Residual
Enhancement
|
Transmission Map
|
Convolution Neural Network
|
Ground Truth
|
Table XV. Dataset
Used Details
Dataset
|
O-Haze[17]
|
Single Image
Haze Removal Using Dark Channel Prior.
[18]
|
Statement
|
It is a collection of 40
outdoor hazy-ground truth images
|
Canon.png has been taken from
[18].
|
[1] P. S. Chavez
(Jr.), An improved dark-object subtraction technique for atmospheric scattering
correction of multispectral data, Remote Sensing and Environment, Elsevier,
vol-24(3), pp-459-479,1988
[2]. P. Oakley and B. L.
Satherley, “Improving image quality in poor visibility conditions using a
physical model for contrast degradation,” IEEE Trans. Image Process., vol. 7,
no. 2, pp. 167-179, Feb. 1998.
[3] Simonyan, K., and A.
Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition.
Computational and Biological Learning Society, 2015, pp. 1–14.
[4] J.
Kim, J. K. Lee and K. M. Lee, "Accurate Image Super-Resolution
Using Very Deep Convolutional Networks," 2016 IEEE Conference on Computer
Vision and Pattern Recognition (CVPR), 2016, pp. 1646-1654, doi:
10.1109/CVPR.2016.182.
[5]
W. Yang, X. Zhang, Y. Tian, W. Wang, J. Xue and Q. Liao, "Deep
Learning for Single Image Super-Resolution: A Brief Review," in IEEE
Transactions on Multimedia, vol. 21, no. 12, pp. 3106-3121, Dec. 2019, doi:
10.1109/TMM.2019.2919431.
[6] D. Das, S. S. Chaudhuri and
S. Roy, "Dehazing technique based on dark channel prior model with sky
masking and its quantitative analysis," 2016 2nd International Conference
on Control, Instrumentation, Energy & Communication (CIEC), 2016, pp.
207-210, doi: 10.1109/CIEC.2016.7513741.
[7] Sangita Roy and Sheli Sinha
Chaudhuri, “Fast Single Image Haze Removal Scheme Using Self-Adjusting:
Haziness Factor Evaluation”, International Journal of Virtual and Augmented
Reality (IJVAR), 3 (1), 2019, pp. 42-57.
[8] Sangita Roy and Sheli Sinha
Chaudhuri, “WLMS-based Transmission Refined Self-Adjusted No Reference Weather
Independent Image Visibility Improvement”, IETE Journal of Research, September
2020. https://doi.org/10.1080/03772063.2019.1662335
.[9] S Roy, S S Chaudhuri, Low
Complexity Single Color Image Dehazing Technique, Intelligent Multidimensional
Data and Image Processing, IGI Global,2018(special session).
[10] H. Koschmieder, Theorie
der horizontalen sichtweite, Beitr.Phys. Freien Atm., vol. 12, 1924, pp.
171–181.
[11] E J McCartney, Optics of
the Atmosphere: Scattering by Molecules and Particles, New York, NY, USA:Wiley,
1976.
[12]
He, Kaiming, et al. "Delving deep into rectifiers: Surpassing
human-level performance on imagenet classification."
Proceedings
of the IEEE international conference on computer vision. 2015.
[13] Zhou Wang, A. C. Bovik, H.
R. Sheikh and E. P. Simoncelli, "Image quality assessment: from error
visibility to structural similarity," in IEEE Transactions on Image
Processing, vol. 13, no. 4, pp. 600-612, April 2004, doi: 10.1109/TIP.2003.819861.
[14] Sara, U., Akter, M. and
Uddin, M. (2019) Image Quality Assessment through FSIM, SSIM, MSE and PSNR—A
Comparative Study. Journal of Computer and Communications,
7,
8-18. doi:
10.4236/jcc.2019.73002.
[15] A. Mittal, A. K. Moorthy
and A. C. Bovik, "No-Reference Image Quality Assessment in the Spatial
Domain," in IEEE Transactions on Image Processing, vol. 21, no. 12, pp.
4695-4708, Dec. 2012, doi: 10.1109/TIP.2012.2214050.
[16] Mittal, A., R. Soundararajan,
and A. C. Bovik. "Making a Completely Blind Image Quality
Analyzer." IEEE Signal Processing Letters. Vol. 22, Number 3, March
2013, pp. 209–212.
[17]
C. O. Ancuti, C. Ancuti, R. Timofte and C. De Vleeschouwer,
"O-HAZE: A Dehazing Benchmark with Real Hazy and Haze-Free Outdoor
Images," 2018 IEEE/CVF Conference on Computer Vision and Pattern
Recognition Workshops (CVPRW), 2018, pp. 867-8678, doi:
10.1109/CVPRW.2018.00119.
[18]
K. He, J. Sun and X. Tang, "Single Image Haze Removal Using Dark
Channel Prior," in
IEEE Transactions on Pattern Analysis and
Machine Intelligence, vol.
33, no. 12, pp.
2341-2353, Dec. 2011, doi: 10.1109/TPAMI.2010.168.
[19] G. Meng, Y. Wang, J. Duan,
S. Xiang, and C. Pan. Efficient image dehazing with boundary constraint and
contextual regularization. In IEEE Int. Conf. on Computer Vision, 2013.
[20] R. Fattal. Dehazing using
color-lines. ACM Trans. on Graph., 2014.
[21] D. Berman, T. Treibitz,
and S. Avidan. Non-local image dehazing. IEEE Intl. Conf. Comp. Vision, and
Pattern Recog, 2016.
[22]
Ren W., Liu S., Zhang H., Pan J., Cao X., Yang MH. (2016) Single Image
Dehazing via Multi-scale Convolutional Neural Networks. In: Leibe B., Matas J.,
Sebe N., Welling M. (eds) Computer Vision – ECCV 2016. ECCV 2016. Lecture Notes
in Computer Science, vol 9906. Springer, Cham.
https://doi.org/10.1007/978-3-319-46475-6_10
[23]
W. Yang, X. Zhang, Y. Tian, W. Wang, J. Xue and Q. Liao, "Deep
Learning for Single Image Super-Resolution: A Brief Review," in IEEE
Transactions on Multimedia, vol. 21, no. 12, pp. 3106-3121, Dec. 2019, doi:
10.1109/TMM.2019.2919431.
[24] C. Dong,
C. C. Loy, K. He, and X. Tang, “Learning a deep convolutional network for image
super-resolution,” in Proc. Eur. Conf. Comput. Vis., 2014, pp. 184–199.
[25] C. Dong,
C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional
networks,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 2, pp.
295–307, Feb. 2016.
[26] A. S.
Parihar, Y. K. Gupta, Y. Singodia, V. Singh and K. Singh, "A Comparative
Study of Image Dehazing Algorithms," 2020 5th International
Conference on Communication and Electronics Systems (ICCES), 2020, pp. 766-771,
doi: 10.1109/ICCES48766.2020.9138037.
[27] W.
Yang et al., "Advancing Image Understanding in Poor Visibility
Environments: A Collective Benchmark Study," in IEEE Transactions on
Image Processing, vol. 29, pp. 5737-5752, 2020, doi: 10.1109/TIP.2020.2981922.
[28] C.
Chengtao, Z. Qiuyu and L. Yanhua, "A survey of image dehazing
approaches," The 27th Chinese Control and Decision Conference (2015 CCDC),
2015, pp. 3964-3969, doi: 10.1109/CCDC.2015.7162616.
[29] B. Cai,
X. Xu, K. Jia, C. Qing and D. Tao, "DehazeNet: An End-to-End System for
Single Image Haze Removal," in IEEE Transactions on Image Processing, vol.
25, no. 11, pp. 5187-5198, Nov. 2016, doi: 10.1109/TIP.2016.2598681.
[30] Ph.D. Thesis, Sangita Roy,
Development of Improved Visibility Restoration Techniques using Various
Intensity Parameter Tuning, ETCE Department, Jadavpur University, July 2021.
[31] J. -B. Huang, A. Singh and
N. Ahuja, "Single image super-resolution from transformed
self-exemplars," 2015 IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), 2015, pp. 5197-5206, doi: 10.1109/CVPR.2015.7299156.
[32] J. Yang, J. Wright, T. S.
Huang and Y. Ma, "Image Super-Resolution Via Sparse Representation,"
in IEEE Transactions on Image Processing, vol. 19, no. 11, pp. 2861-2873, Nov.
2010, doi: 10.1109/TIP.2010.2050625..
[33] G. Sharma, W. Wu, and E.
Dalal. The ciede2000 color-difference formula: Implementation notes, sup-
plementary test data, and mathematical observations.
Color Research and
Applications, 2005,
https://doi.org/10.1002/col.20070.
[34] S Westland, C
Ripamonte, Computational Colour Science using MATLAB, Wiley-IS&T Series in
Imaging Science and Technology.
[35]
Y. Y. Schechner, S. G. Narasimhan and S. K. Nayar, "Instant
dehazing of images using polarization,"
Proceedings of the
2001 IEEE Computer Society Conference on Computer Vision and Pattern
Recognition. CVPR 2001,
2001, pp. I-I, doi:
10.1109/CVPR.2001.990493.
[36] S. G. Narasimhan and S. K. Nayar,
"Chromatic framework for vision in bad weather,"
Proceedings IEEE
Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat.
No.PR00662),
2000, pp. 598-605 vol.1, doi:
10.1109/CVPR.2000.855874.
[37]
CIE. Improvement to industrial colour-difference
evaluation. Vienna:CIE Publication No. 142-2001, Central Bureau of the CIE;
2001.
[38] R. T. Tan, "Visibility in bad weather from a single
image," 2008 IEEE Conference on Computer Vision and Pattern Recognition,
2008, pp. 1-8, doi: 10.1109/CVPR.2008.4587643.
[39]
Kopf, Johannes, et al.
"Deep photo: Model-based photograph enhancement and viewing."
ACM
transactions on graphics (TOG)
27.5
(2008): 1-10.
[40] S Roy, S S Chaudhuri, Single Image Very Deep Super Resolution
(SIVDSR) Dehazing, BECITHCON 2021, 4-5 December 2021.
[41] S Roy, Single Image DnCNN Visibility Improvement
(SImDnCNNVI),
Scientific Visualization, volume 14,
number 3, pages 92 – 106, September 19th
2022,
https://doi.org/10.26583/sv.14.3.07.