Intro:
On the current page, I’m going to explain how to write an “edge detection” core using the “Sobel method”. Our purpose is to implement this system on FPGA; so we should program it by using one of the Hardware Design Languages like Verilog, VHDL, etc. Here you must have at least basic knowledge about HDL.
In this project we used Verilog. Note that you can write the core and of course all the systems by using one of the languages that you are familiar with.
Basic Info:
Sobel operator: [α]
The Sobel operator, sometimes called Sobel Filter, is used in image processing and computer vision, particularly within edge detectionalgorithms, and creates an image which emphasizes edges and transitions. It is named after Irwin Sobel, who presented the idea of an “Isotropic 3×3 Image Gradient Operator” at a talk at the Stanford Artificial Intelligence Project (SAIP) in 1968.[1] Technically, it is a discrete differentiation operator, computing an approximation of the gradient of the image intensity function. At each point in the image, the result of the Sobel operator is either the corresponding gradient vector or the norm of this vector. The Sobel operator is based on convolving the image with a small, separable, and integer valued filter in horizontal and vertical direction and is therefore relatively inexpensive in terms of computations …
Suppose that A is an image then using Sobel operator we have:
Gy=
and Gx=To obtain the value of G we must calculate the following equation. So G=
or
We can calculate the angle: (but we don’t need it in the current project)
Impact of Sobel filter:
As I mentioned, Sobel is one of the methods that we can apply to an image to obtain edges. Let’s see how:
[β]
Applying Sobel filter and calculating Gy and Gx formulas (based on image pixels):
First, we should take a look and understand how this filter works, then we calculate and determine a couple of formulas that we need them in the programming section.
As you can see above, the “x” filter matrix equals the matrix that we have talked about it before. The filter will be applied to the image by multiplying a 3×3 filter matrix to a 3×3 blocks of pixels. And then this process continues by sweeping all of the images in 3×3 blocks.
Note that there is some lost data. If we look carefully we can find out that we lose pixels located on the end of image corners (Top, Bottom, Left, Right). This occurs by applying the Sobel filter because after calculating G, there is only one value instead of 9 values and we should write it in the place of 5th element of the block as a result of the operation. So our output image must have 2 rows and 2 columns less than the primary image. For instance if the primary image resolution is 640×480, the result image should be 638×478.
With keeping these notes in mind, now we can calculate the parametric formulas required for applying the Sobel filter:
Gx[i, j] = p[i+1, j-1] + 2 × p[i+1, j] + p(i+1, j+1) – { p[i-1, j-1] + 2 × p[i-1, j] + p(i-1, j+1) }
and
Gy[i, j] = p[i-1, j+1] + 2 × p[i, j+1] + pi+1, j+1) – { p[i-1, j-1] + 2 × p[i, j-1] + p(i+1, j-1) }
Well, now as you can see there is no p[i,j] in above formulas. Because this element multiplied to 0 according to Gy and Gx matrices.
Hardware design:
Sobel core:
As I said before I prefer Verilog to implement the core using the above formulas. General and detailed (inner) block diagrams of the core, must be as below:
Note that the input and output are between 0 to 255 and the core must have 8 inputs and one output line by the width of 8 bits.
In the inner diagram the output of the adder has 11 bits. One for sign and others for the output of 4 adder blocks (4 × 255).
General configuration diagram of the system:
Below figure indicates the general configuration of the system.
The system operates as follows:
- A sample image is saved as an 8-bit grayscale image
- The image is converted to RAW data array and saved as a text file.
- The “Testbench” program reads the data and copies them to RAM modules implemented on FPGA
- The data is read and written to a 3 × 3 RAM module by a valid 3 × 3 data generation matrix.
- The data is transmitted to Sobel core and the output is saved as a txt file
- The output is converted to image
- The result image is visible
Diagram of the system:
This image shows all components of the implemented system:
Description of RAM module functions:
In this system two RAM modules have been used; one of them for writing the whole 8-bit pixels data to it and the other for valid 3 × 3 data generation. Both of these modules have been implemented by utilizing FPGA’s own resources.
The first module is the same size as RAW data, therefore this is a 3D array with the size of m × n and depth of 8 bit.
The second is a 3 × 3 memory module which has the same bit depth as RAW data and used to transiting data to the Sobel core.
After writing the whole data to the first RAM module, programming methods have been used to sweeping this module and then generating valid 3 × 3 arrays.
You can observe how the aforementioned tasks are done in the next image.
Programming:
By determining all concepts of the project, Now we can program and run what we have designed.
We used Linux, its application, and software like “GIMP” for saving 8-bit grayscale images, Matlab for converting images to RAW data and vice versa, and “Modelsim” for simulating and observing the result.
Project information:
The resolution of the 8-bit grayscale image is 640 × 480, so the first RAM module size is 307200 bytes.
The below table shows the file names and description of each file:
Filename | Description |
core_sobel.v | Verilog code for Sobel core hardware implementation |
mem_sobel.v | implementation of first ram module |
main_sobel.v | main module witch has a valid 3 × 3 data generator |
testbench_sobel.v | test bench module for initialization, control, and simulation |
top.v | top module |
data1.txt | input image raw data pixel |
result_img.txt | output raw data |
sbl.jpg | input image file |
result_img.txt | output image file |
You can download all files from the attachment at the end of this page.
Well, we are going to explain how to install the required applications and using them.
The version of Mentor Graphics Modelsim used in this project is 10.1c under Linux. Note that you have to install some important packages like GCC, java runtime environment, and some others before you begin to install software like Modelsim, Matlab, etc.
I prefer to use the Command line rather than working with GUI! so I used Linux “Terminal” to run the whole project.
1) Converting 8-bit grayscale image to RAW data via MATLAB (or similar tools):
mahi@mahi-VPCF133FX:~$ matlab -nodisplay >> RGB = imread('sbl.jpg'); >> I = rgb2gray(RGB); >> dlmwrite('data.txt',I,'delimiter',' ');
The original image used in the project:
In below shoot you can see a text file which contains RAW data of above image file:
2) Compiling and simulation of the project in Modelsim:
mahi@mahi-VPCF133FX:~$ cd Desktop/edge_detection/sim mahi@mahi-VPCF133FX:~/Desktop/edge_detection/sim$ vlog ../hdl/*.v Model Technology ModelSim SE-64 vlog 10.1c Compiler 2012.07 Jul 27 2012 -- Compiling module core_sobel -- Compiling module main_sobel -- Compiling module mem_sobel -- Compiling module testbench_sobel -- Compiling module Top Top level modules: Top mahi@mahi-VPCF133FX:~/Desktop/$ edge_detection/sim mahi@mahi-VPCF133FX:~/Desktop/edge_detection/sim$ vsim -c -novopt Top Reading /home/mahi/modelsim/modeltech/tcl/vsim/pref.tcl # 10.1c # vsim -c -novopt Top # Refreshing /home/mahi/Desktop/edge_detection/sim/work.Top # Loading work.Top # Refreshing /home/mahi/Desktop/edge_detection/sim/work.testbench_sobel # Loading work.testbench_sobel # Refreshing /home/mahi/Desktop/edge_detection/sim/work.mem_sobel # Loading work.mem_sobel # Refreshing /home/mahi/Desktop/edge_detection/sim/work.main_sobel # Loading work.main_sobel # Refreshing /home/mahi/Desktop/edge_detection/sim/work.core_sobel # Loading work.core_sobel VSIM 1> VSIM 1> run 12ms VSIM 2>
Now we must get the result txt file that contains result RAW data. It looks to:
3) Converting result RAW data to a visible “result” image:
mahi@mahi-VPCF133FX:~$ matlab -nodisplay >> M = dlmread(result_img.txt); >> imwrite(M,'result_img.jpg');
And finally, our real visible result file created and we can see detected edges:
Note that we lost some data (the size length and height of the image) as I explained in the first section.
Download source code: DOWNLOAD
Update 2024 – Disclaimer: I received some reports that people have problems/errors using the code (simulation, etc). Please note that:
- This code is only to demonstrate the design and steps that were done in the project.
- No IDE is used.
- A pro version of ModelSim is used.
- The published code does not include the project for any specific FPGA device.
- No support can be provided.
[α]: Wikipedia [β]: Carnegie Mellon School of Computer Science