“Sobel” method simulation and implementation on FPGA

Intro:

On the current page, I’m going to explain how to write an “edge detection” core using the “Sobel method”. Our purpose is to implement this system on FPGA; so we should program it by using one of the Hardware Design Languages like Verilog, VHDL, etc. Here you must have at least basic knowledge about HDL.

In this project we used Verilog. Note that you can write the core and of course all the systems by using one of the languages that you are familiar with.

Basic Info:

Sobel operator: [α]

The Sobel operator, sometimes called Sobel Filter, is used in image processing and computer vision, particularly within edge detectionalgorithms, and creates an image which emphasizes edges and transitions. It is named after Irwin Sobel, who presented the idea of an “Isotropic 3×3 Image Gradient Operator” at a talk at the Stanford Artificial Intelligence Project (SAIP) in 1968.[1] Technically, it is a discrete differentiation operator, computing an approximation of the gradient of the image intensity function. At each point in the image, the result of the Sobel operator is either the corresponding gradient vector or the norm of this vector. The Sobel operator is based on convolving the image with a small, separable, and integer valued filter in horizontal and vertical direction and is therefore relatively inexpensive in terms of computations …

Suppose that A is an image then using Sobel operator we have:

Gy=

and Gx=To obtain the value of G we must calculate the following equation. So G=

image006

or

image008We can calculate the angle: (but we don’t need it in the current project)image010

 Impact of Sobel filter:

As I mentioned, Sobel is one of the methods that we can apply to an image to obtain edges. Let’s see how:

[β]

Primary Image

 

Sobel output

Applying Sobel filter and calculating Gy and Gx formulas (based on image pixels):

First, we should take a look and understand how this filter works, then we calculate and determine a couple of formulas that we need them in the programming section.

sbll

As you can see above, the “x” filter matrix equals the matrix that we have talked about it before. The filter will be applied to the image by multiplying a 3×3 filter matrix to a 3×3 blocks of pixels. And then this process continues by sweeping all of the images in 3×3 blocks.

Note that there is some lost data. If we look carefully we can find out that we lose pixels located on the end of image corners (Top, Bottom, Left, Right). This occurs by applying the Sobel filter because after calculating G, there is only one value instead of 9 values and we should write it in the place of  5th element of the block as a result of the operation. So our output image must have 2 rows and 2 columns less than the primary image. For instance if the primary image resolution is 640×480, the result image should be 638×478.

With keeping these notes in mind, now we can calculate the parametric formulas required for applying the Sobel filter:

Gx[i, j] = p[i+1, j-1] + 2 × p[i+1, j] + p(i+1, j+1) – { p[i-1, j-1] + 2 × p[i-1, j] + p(i-1, j+1) }

and

Gy[i, j] = p[i-1, j+1] + 2 × p[i, j+1] + pi+1, j+1) – { p[i-1, j-1] + 2 × p[i, j-1] + p(i+1, j-1) }

Well, now as you can see there is no p[i,j] in above formulas. Because this element multiplied to 0 according to Gy and Gmatrices.

Hardware design:

Sobel core:

As I said before I prefer Verilog to implement the core using the above formulas. General and detailed (inner) block diagrams of the core, must be as below:

Note that the input and output are between 0 to 255 and the core must have 8 inputs and one output line by the width of 8 bits.

Diagram_1
General diagram of the core
diagram
Detailed diagram of the core

In the inner diagram the output of the adder has 11 bits. One for sign and others for the output of 4 adder blocks (4 × 255).

General configuration diagram of the system:

Below figure indicates the general configuration of the system.

System configuration
System configuration

The system operates as follows:

  1. A sample image is saved as an 8-bit grayscale image
  2. The image is converted to RAW data array and saved as a text file.
  3. The “Testbench” program reads the data and copies them to RAM modules implemented on FPGA
  4. The data is read and written to a 3 × 3 RAM module by a valid 3 × 3 data generation matrix.
  5. The data is transmitted to Sobel core and the output is saved as a txt file
  6. The output is converted to image
  7. The result image is visible

Diagram of the system:

This image shows all components of the implemented system:

System diagram
System diagram

Description of RAM module functions:

In this system two RAM modules have been used; one of them for writing the whole 8-bit pixels data to it and the other for valid 3 × 3 data generation. Both of these modules have been implemented by utilizing FPGA’s own resources.

The first module is the same size as RAW data, therefore this is a 3D array with the size of m × n and depth of 8 bit.

The second is a  3 × 3 memory module which has the same bit depth as RAW data and used to transiting data to the Sobel core.

After writing the whole data to the first RAM module, programming methods have been used to sweeping this module and then generating valid 3 × 3 arrays.

You can observe how the aforementioned tasks are done in the next image.

RAM
RAM modules construction and function

Programming:

By determining all concepts of the project, Now we can program and run what we have designed.

We used Linux, its application, and software like “GIMP” for saving 8-bit grayscale images, Matlab for converting images to RAW data and vice versa, and “Modelsim” for simulating and observing the result.

Project information:

The resolution of the 8-bit grayscale image is  640 × 480, so the first RAM module size is 307200 bytes.

The below table shows the file names and description of each file:

Filename Description
core_sobel.v Verilog code for Sobel core hardware implementation
mem_sobel.v implementation of first ram module
main_sobel.v main module witch has a valid 3 × 3 data generator
testbench_sobel.v test bench module for initialization, control, and simulation
top.v top module
data1.txt input image raw data pixel
result_img.txt output raw data
sbl.jpg input image file
result_img.txt output image file

You can download all files from the attachment at the end of this page.

Well, we are going to explain how to install the required applications and using them.

The version of Mentor Graphics Modelsim used in this project is 10.1c under Linux. Note that you have to install some important packages like GCC, java runtime environment, and some others before you begin to install software like Modelsim, Matlab, etc.

I prefer to use the Command line rather than working with GUI! so I used Linux “Terminal” to run the whole project.

1) Converting 8-bit grayscale image to RAW data via MATLAB (or similar tools):

mahi@mahi-VPCF133FX:~$ matlab -nodisplay
>> RGB = imread('sbl.jpg');
>> I = rgb2gray(RGB);
>> dlmwrite('data.txt',I,'delimiter',' ');

The original image used in the project:

640 x 480 8 bit gray scale image
640 x 480 8 bit grayscale image

In below shoot you can see a text file which contains RAW data of above image file:

RAW data txt file
RAW data txt file

2) Compiling and simulation of the project in Modelsim:

mahi@mahi-VPCF133FX:~$ cd Desktop/edge_detection/sim

mahi@mahi-VPCF133FX:~/Desktop/edge_detection/sim$ vlog ../hdl/*.v
 Model Technology ModelSim SE-64 vlog 10.1c Compiler 2012.07 Jul 27
 2012
 -- Compiling module core_sobel
 -- Compiling module main_sobel
 -- Compiling module mem_sobel
 -- Compiling module testbench_sobel
 -- Compiling module Top
 Top level modules:
 Top
 mahi@mahi-VPCF133FX:~/Desktop/$ edge_detection/sim

mahi@mahi-VPCF133FX:~/Desktop/edge_detection/sim$ vsim -c -novopt Top
 Reading /home/mahi/modelsim/modeltech/tcl/vsim/pref.tcl
 # 10.1c
 # vsim -c -novopt Top
 # Refreshing /home/mahi/Desktop/edge_detection/sim/work.Top
 # Loading work.Top
 # Refreshing /home/mahi/Desktop/edge_detection/sim/work.testbench_sobel
 # Loading work.testbench_sobel
 # Refreshing /home/mahi/Desktop/edge_detection/sim/work.mem_sobel
 # Loading work.mem_sobel
 # Refreshing /home/mahi/Desktop/edge_detection/sim/work.main_sobel
 # Loading work.main_sobel
 # Refreshing /home/mahi/Desktop/edge_detection/sim/work.core_sobel
 # Loading work.core_sobel
 VSIM 1>

VSIM 1> run 12ms
 VSIM 2>

Now we must get the result txt file that contains result RAW data. It looks to:

Output RAW data
Output RAW data

3) Converting result RAW data to a visible “result” image:

mahi@mahi-VPCF133FX:~$ matlab -nodisplay
>> M = dlmread(result_img.txt);
>> imwrite(M,'result_img.jpg');

And finally, our real visible result file created and we can see detected edges:

638 x 478 Result image
638 x 478 Result image

Note that we lost some data (the size length and height of the image) as I explained in the first section.


Download  source code: DOWNLOAD

Update 2024 – Disclaimer: I received some reports that people have problems/errors using the code (simulation, etc). Please note that:

  • This code is only to demonstrate the design and steps that were done in the project.
  • No IDE is used.
  • A pro version of ModelSim is used.
  • The published code does not include the project for any specific FPGA device.
  • No support can be provided.

[α]: Wikipedia

[β]: Carnegie Mellon School of Computer Science