“Sobel” method simulation and implementation on FPGA

Posted by in Hellotech

Intro:

In the current page, I’m going to explain how to write an “edge detection” core using “Sobel method”. Our purpose is to implement this system on FPGA; so we should program it by using one of the Hardware Design Languages like Verilog, VHDL, etc. Here you must have at least basic knowledge about HDL.

In this project we used Verilog. Note that you can write the core and of course all the system by using which one of languages that you are familiar.

Basic Info:

Sobel operator: [α]

The Sobel operator, sometimes called Sobel Filter, is used in image processing and computer vision, particularly within edge detectionalgorithms, and creates an image which emphasizes edges and transitions. It is named after Irwin Sobel, who presented the idea of an “Isotropic 3×3 Image Gradient Operator” at a talk at the Stanford Artificial Intelligence Project (SAIP) in 1968.[1] Technically, it is a discrete differentiation operator, computing an approximation of the gradient of the image intensity function. At each point in the image, the result of the Sobel operator is either the corresponding gradient vector or the norm of this vector. The Sobel operator is based on convolving the image with a small, separable, and integer valued filter in horizontal and vertical direction and is therefore relatively inexpensive in terms of computations …

Suppose that A is an image then using Sobel operator we have:

Gy=

and Gx=To obtain the value of G we must calculate the fallowing equation. So G=

image006

or

image008We can calculate the angle: (but we don’t need it in current project)image010

 Impact of Sobel filter:

As I mentioned, Sobel is one of the methods that we can apply on an images to obtain edges. Let’s see how:

[β]

Primary Image

 

Sobel output

Applying Sobel filter and calculating Gy and Gx formulas (based on image pixels):

First we should take a look and understand that how this filter works, then we calculate and determine couples of formulas that we need them in programming section.

sbll

As you can see above, the “x” filter matrix is equals to the matrix that we have talked about it before. The filter will be applied to the image by multiplying a 3×3 filter matrix to a 3×3 blocks of pixels. And then this process continues by sweeping all of the image in 3×3 blocks.

Note that there is some lost data. If we look carefully we can find out that we lose pixels located on end of image corners (Top, Bottom, Left, Right). This occures by applying the Sobel filter because after calculating G, there is only one value instead of 9 values and we should write it in the place of  5th element of the block as a result of operation. So our output image must have 2 rows and 2 columns less than primary image. For instance if the primary image resolution is 640×480, the result image should be 638×478.

With keeping these notes in mind, now we can calculate the parametric formulas required for applying the Sobel filter:

Gx[i, j] = p[i+1, j-1] + 2 × p[i+1, j] + p(i+1, j+1) – { p[i-1, j-1] + 2 × p[i-1, j] + Im(i-1, j+1) }

and

Gy[i, j] = p[i-1, j+1] + 2 × p[i, j+1] + pi+1, j+1) – { p[i-1, j-1] + 2 × p[i, j-1] + p(i+1, j-1) }

Well, Now as you can see there is no p[i,j] in above formulas. Because this element multiplied to 0 according to Gy and Gmatrices.

Hardware design:

Sobel core:

As I said before I prefer Verilog to implement the core using above formulas. General and detailed (inner) block diagrams of the core, must be as below:

Note that the input and output are between 0 to 255 and the core must have 8 inputs and one output line by the width of 8 bits.

Diagram_1

General diagram of the core

diagram

Detailed diagram of the core

In the inner diagram the output of the adder has 11 bits. One for sign and others for the output of 4 adder blocks (4 × 255).

General configuration diagram of the system:

Below figure indicates the general configuration of the system.

System configuration

System configuration

The system operates as fallows:

  1. A sample image is saved as a 8 bit gray scale image
  2. The image is converted to RAW data array and saved as a txt file.
  3. The “Testbench” program reads the data and copies them to RAM modules implemented on FPGA
  4. The data is read and written to a 3 × 3 RAM module by a valid 3 × 3 data generation matrix.
  5. The data is transmitted to Sobel core and the output is saved as a txt file
  6. The output is converted to image
  7. The result image is visible

Diagram of the system:

This image shows all components of the implemented system:

System diagram

System diagram

Description of RAM module functions:

In this system two RAM modules have been used; One of them for writing the whole 8 bit pixels data to it and the other for valid 3 × 3 data generation. Both of these modules have been implemented by utilizing of FPGA’s own resources.

The first module is the same size as RAW data, therefore this is a 3D array with the size of  m × n and depth of 8 bit.

The second is a  3 × 3 memory module witch has same bit depth as RAW data and used to transiting data to the Sobel core.

After writing the whole data to first RAM module, programming methods has been used to sweeping this module and then generating valid 3 × 3 arrays.

You can observe how aforementioned tasks done in the next image.

RAM

RAM modules construction and function

Programming:

By the determining all concepts of the project, Now we can program and run what we have designed.

We used Linux platform and its application and software like “GIMP” for saving 8 bit gray scale images, “imtools” for converting images to RAW data and vice versa, and “Modelsim” for simulating and observing the result.

Project information:

The resolution of the 8 bit gray scale image is  640 × 480, so the first RAM module size is 307200 bytes.

The below table shows the file names and description of each file:

File name Description
core_sobel.v verilog code for sobel core hardware implementation
mem_sobel.v implementation of first ram module
main_sobel.v main module witch has valid 3 × 3 data generator
testbench_sobel.v test bench module for initialization, control nd simulation
top.v top module
data1.txt input image raw data pixel
result_img.txt output raw data
sbl.jpg input image file
result_img.txt result image file

You can download all files from attachment in the end of this page.

Well, we are going to explain how to install required applications and using them.

The version of Mentor Graphics Modelsim used in this project is 10.1c under Linux platform. Note that you have to install some important packages like gcc, java runtime environment, libgpeg, libpng and some others before than you begin to install softwares like Modelsim, imtools and …

I prefer to use Command line rather than working with GUI! so I used powerful tool called “Terminal” to run whole project.

1) Converting 8 bit gray scale image to RAW data via imtool:

[email protected]:~$ cd Desktop/imtools

[email protected]:~/Desktop/imtools$ ./imtool sbl.jpg COPY data1.txt
 Reading JPEG sbl.jpg
 JPEG File Information:
 Image width and height: 640 pixels and 480 pixels.
 Color components per pixel: 1.
 Color space: 1.
 Writing RAW data1.txt
 [email protected]:~/Desktop/imtools$

The original image that used in the project:

640 x 480 8 bit gray scale image

640 x 480 8 bit gray scale image

In below shoot you can see a txt file witch contains RAW data of above image file:

RAW data txt file

RAW data txt file

2) Compiling and simulation of project in Modelsim:

[email protected]:~$ cd Desktop/edge_detection/sim

[email protected]:~/Desktop/edge_detection/sim$ vlog ../hdl/*.v
 Model Technology ModelSim SE-64 vlog 10.1c Compiler 2012.07 Jul 27
 2012
 -- Compiling module core_sobel
 -- Compiling module main_sobel
 -- Compiling module mem_sobel
 -- Compiling module testbench_sobel
 -- Compiling module Top
 Top level modules:
 Top
 [email protected]:~/Desktop/$ edge_detection/sim

[email protected]:~/Desktop/edge_detection/sim$ vsim -c -novopt Top
 Reading /home/mahi/modelsim/modeltech/tcl/vsim/pref.tcl
 # 10.1c
 # vsim -c -novopt Top
 # // ModelSim SE-64 10.1c Jul 27 2012 Linux 3.13.0-24-generic
 # //
 # // Copyright 1991-2012 Mentor Graphics Corporation
 # // All Rights Reserved.
 # //
 # // THIS WORK CONTAINS TRADE SECRET AND PROPRIETARY INFORMATION
 # // WHICH IS THE PROPERTY OF MENTOR GRAPHICS CORPORATION OR ITS
 # // LICENSORS AND IS SUBJECT TO LICENSE TERMS.
 # //
 # Refreshing /home/mahi/Desktop/edge_detection/sim/work.Top
 # Loading work.Top
 # Refreshing /home/mahi/Desktop/edge_detection/sim/work.testbench_sobel
 # Loading work.testbench_sobel
 # Refreshing /home/mahi/Desktop/edge_detection/sim/work.mem_sobel
 # Loading work.mem_sobel
 # Refreshing /home/mahi/Desktop/edge_detection/sim/work.main_sobel
 # Loading work.main_sobel
 # Refreshing /home/mahi/Desktop/edge_detection/sim/work.core_sobel
 # Loading work.core_sobel
 VSIM 1>

VSIM 1> run 12ms
 VSIM 2>

Now we must get the result txt file that contains result RAW data. It looks to:

Output RAW data

Output RAW data

3) Converting result RAW data to visible “result” image:

[email protected]:~$ cd Desktop/imtools

[email protected]:~/Desktop/imtools$ ./imtool result_img.txt COPY result_img.jpg
 Reading RAW result_img.txt
 Writing JPEG result_img.jpg
 [email protected]:~/Desktop/imtools$

And finally our real visible result file created and we can see detected edges:

638 x 478 Result image

638 x 478 Result image

Note that we lost some data (the size of length and height of image) as I explained in first section.

All the vital subjects have been explained, if you have any problem in any procedure of the project, please feel free to contact me by mail or other social networks. I’ll try to respond as soon as possible. You’ll find this information on Get in touch section.


Download the code: DOWNLOAD


[α]: Wikipedia

[β]: Carnegie Mellon School of Computer Science