“Sobel” method simulation and implementation on FPGA

Posted by in Hellotech

Intro:

In the current page, I’m going to explain how to write an “edge detection” core using “Sobel method”. Our purpose is to implement this system on FPGA; so we should program it by using one of the Hardware Design Languages like Verilog, VHDL, etc. Here you must have at least basic knowledge about HDL.

In this project we used Verilog. Note that you can write the core and of course all the system by using which one of languages that you are familiar.

Basic Info:

Sobel operator: [α]

The Sobel operator, sometimes called Sobel Filter, is used in image processing and computer vision, particularly within edge detectionalgorithms, and creates an image which emphasizes edges and transitions. It is named after Irwin Sobel, who presented the idea of an “Isotropic 3×3 Image Gradient Operator” at a talk at the Stanford Artificial Intelligence Project (SAIP) in 1968.[1] Technically, it is a discrete differentiation operator, computing an approximation of the gradient of the image intensity function. At each point in the image, the result of the Sobel operator is either the corresponding gradient vector or the norm of this vector. The Sobel operator is based on convolving the image with a small, separable, and integer valued filter in horizontal and vertical direction and is therefore relatively inexpensive in terms of computations …

Suppose that A is an image then using Sobel operator we have:

Gy=

and Gx=To obtain the value of G we must calculate the fallowing equation. So G=

or

We can calculate the angle: (but we don’t need it in current project)

Impact of Sobel filter:

As I mentioned, Sobel is one of the methods that we can apply on an images to obtain edges. Let’s see how:

[β]

Primary Image

Sobel output

Applying Sobel filter and calculating Gy and Gx formulas (based on image pixels):

First we should take a look and understand that how this filter works, then we calculate and determine couples of formulas that we need them in programming section.

As you can see above, the “x” filter matrix is equals to the matrix that we have talked about it before. The filter will be applied to the image by multiplying a 3×3 filter matrix to a 3×3 blocks of pixels. And then this process continues by sweeping all of the image in 3×3 blocks.

Note that there is some lost data. If we look carefully we can find out that we lose pixels located on end of image corners (Top, Bottom, Left, Right). This occures by applying the Sobel filter because after calculating G, there is only one value instead of 9 values and we should write it in the place of  5th element of the block as a result of operation. So our output image must have 2 rows and 2 columns less than primary image. For instance if the primary image resolution is 640×480, the result image should be 638×478.

With keeping these notes in mind, now we can calculate the parametric formulas required for applying the Sobel filter:

and

Gy[i, j] = p[i-1, j+1] + 2 × p[i, j+1] + pi+1, j+1) – { p[i-1, j-1] + 2 × p[i, j-1] + p(i+1, j-1) }

Well, Now as you can see there is no p[i,j] in above formulas. Because this element multiplied to 0 according to Gy and Gmatrices.

Hardware design:

Sobel core:

As I said before I prefer Verilog to implement the core using above formulas. General and detailed (inner) block diagrams of the core, must be as below:

Note that the input and output are between 0 to 255 and the core must have 8 inputs and one output line by the width of 8 bits.

General diagram of the core

Detailed diagram of the core

In the inner diagram the output of the adder has 11 bits. One for sign and others for the output of 4 adder blocks (4 × 255).

General configuration diagram of the system:

Below figure indicates the general configuration of the system.

System configuration

The system operates as fallows:

1. A sample image is saved as a 8 bit gray scale image
2. The image is converted to RAW data array and saved as a txt file.
3. The “Testbench” program reads the data and copies them to RAM modules implemented on FPGA
4. The data is read and written to a 3 × 3 RAM module by a valid 3 × 3 data generation matrix.
5. The data is transmitted to Sobel core and the output is saved as a txt file
6. The output is converted to image
7. The result image is visible

Diagram of the system:

This image shows all components of the implemented system:

System diagram

Description of RAM module functions:

In this system two RAM modules have been used; One of them for writing the whole 8 bit pixels data to it and the other for valid 3 × 3 data generation. Both of these modules have been implemented by utilizing of FPGA’s own resources.

The first module is the same size as RAW data, therefore this is a 3D array with the size of  m × n and depth of 8 bit.

The second is a  3 × 3 memory module witch has same bit depth as RAW data and used to transiting data to the Sobel core.

After writing the whole data to first RAM module, programming methods has been used to sweeping this module and then generating valid 3 × 3 arrays.

You can observe how aforementioned tasks done in the next image.

RAM modules construction and function

Programming:

By the determining all concepts of the project, Now we can program and run what we have designed.

We used Linux platform and its application and software like “GIMP” for saving 8 bit gray scale images, “imtools” for converting images to RAW data and vice versa, and “Modelsim” for simulating and observing the result.

Project information:

The resolution of the 8 bit gray scale image is  640 × 480, so the first RAM module size is 307200 bytes.

The below table shows the file names and description of each file:

 File name Description core_sobel.v verilog code for sobel core hardware implementation mem_sobel.v implementation of first ram module main_sobel.v main module witch has valid 3 × 3 data generator testbench_sobel.v test bench module for initialization, control nd simulation top.v top module data1.txt input image raw data pixel result_img.txt output raw data sbl.jpg input image file result_img.txt result image file

Well, we are going to explain how to install required applications and using them.

The version of Mentor Graphics Modelsim used in this project is 10.1c under Linux platform. Note that you have to install some important packages like gcc, java runtime environment, libgpeg, libpng and some others before than you begin to install softwares like Modelsim, imtools and …

I prefer to use Command line rather than working with GUI! so I used powerful tool called “Terminal” to run whole project.

1) Converting 8 bit gray scale image to RAW data via imtool:

```[email protected]:~\$ cd Desktop/imtools

[email protected]:~/Desktop/imtools\$ ./imtool sbl.jpg COPY data1.txt
JPEG File Information:
Image width and height: 640 pixels and 480 pixels.
Color components per pixel: 1.
Color space: 1.
Writing RAW data1.txt
[email protected]:~/Desktop/imtools\$```

The original image that used in the project:

640 x 480 8 bit gray scale image

In below shoot you can see a txt file witch contains RAW data of above image file:

RAW data txt file

2) Compiling and simulation of project in Modelsim:

```[email protected]:~\$ cd Desktop/edge_detection/sim

[email protected]:~/Desktop/edge_detection/sim\$ vlog ../hdl/*.v
Model Technology ModelSim SE-64 vlog 10.1c Compiler 2012.07 Jul 27
2012
-- Compiling module core_sobel
-- Compiling module main_sobel
-- Compiling module mem_sobel
-- Compiling module testbench_sobel
-- Compiling module Top
Top level modules:
Top
[email protected]:~/Desktop/\$ edge_detection/sim

[email protected]:~/Desktop/edge_detection/sim\$ vsim -c -novopt Top
# 10.1c
# vsim -c -novopt Top
# // ModelSim SE-64 10.1c Jul 27 2012 Linux 3.13.0-24-generic
# //
# // Copyright 1991-2012 Mentor Graphics Corporation
# //
# // THIS WORK CONTAINS TRADE SECRET AND PROPRIETARY INFORMATION
# // WHICH IS THE PROPERTY OF MENTOR GRAPHICS CORPORATION OR ITS
# // LICENSORS AND IS SUBJECT TO LICENSE TERMS.
# //
# Refreshing /home/mahi/Desktop/edge_detection/sim/work.Top
# Refreshing /home/mahi/Desktop/edge_detection/sim/work.testbench_sobel
# Refreshing /home/mahi/Desktop/edge_detection/sim/work.mem_sobel
# Refreshing /home/mahi/Desktop/edge_detection/sim/work.main_sobel
# Refreshing /home/mahi/Desktop/edge_detection/sim/work.core_sobel
VSIM 1>

VSIM 1> run 12ms
VSIM 2>```

Now we must get the result txt file that contains result RAW data. It looks to:

Output RAW data

3) Converting result RAW data to visible “result” image:

```[email protected]:~\$ cd Desktop/imtools

[email protected]:~/Desktop/imtools\$ ./imtool result_img.txt COPY result_img.jpg
Writing JPEG result_img.jpg
[email protected]:~/Desktop/imtools\$```

And finally our real visible result file created and we can see detected edges:

638 x 478 Result image

Note that we lost some data (the size of length and height of image) as I explained in first section.

All the vital subjects have been explained, if you have any problem in any procedure of the project, please feel free to contact me by mail or other social networks. I’ll try to respond as soon as possible. You’ll find this information on Get in touch section.

```[α]: Wikipedia