The Needle Recognition algorithm identifies the needle position on the image of analog meter-style gauge. To set up Needle Recognition, acquire at least two full images that show the needle at its minimum and maximum positions.

For every signal, create a new directory under the RecoImages directory in the project folder. Place the full images under the Full_Image directory and follow these steps:

1.Make sure the Camera Transform is set up.

2.In MxTransIt, add the Image Recognition Transform by selecting it from the Toolbox.

3.Configure the Transform properties: Change the Transform name from the default name to the signal name you would like to see in the MxVDev test case. This is the signal name that is added to the signal dictionary.

4.Under Image Recognition Properties, set the Image Recognition Mode to Needle Recognition.

5.Also in properties, set the Base Directory Path. The name of the folder should be the name of the signal.

6.Click the 3dots_button button in Configure Image Recognition to bring up the configuration form for this signal.

7.The first page of the configuration form shows the images found in the Full_Image directory under the Base Directory for this signal. If no images are shown, click cancel and make sure that the directory points to the right Base Directory and that the Full_Image folder contains your full images.

8.Click on the desired Full Image in the list box on the left and draw a rectangle for the ROI around the pattern to be used for Pattern Recognition. Alternatively, you can input the ROI pixel coordinates manually into the fields on the right. For Needle Recognition, the ROI should be the size that will capture the needle in all possible locations. Using the toolbar under the image, you can Zoom In, Zoom Out, and Pan the Image. The Rectangle ROI tool is used to draw or modify the ROI rectangle on the image. You can move the rectangle by clicking in the middle, or drag bounding lines or corners.

 

Set Region of Interest.  Click to expand.

Set Region of Interest.  Click to expand.

 

9.After the ROI is selected, create a Sample image by clicking the Add button. A sample image is a version of the full image cropped to the selected ROI. Sample images are used for testing the search algorithm and Pattern creation. You can add a couple different Samples or click 'Add All' to create a Sample image out of all full images (useful for multi-state signals).
 
Note: Do not click OK. Proceed to the next step.

10.Click the Create Pattern tab to show the just created sample images.

If the needle contrast compared to the background is not high enough, you need to apply image processing. For very bright needles against the dark background (or dark needle on white background) the image processing is not necessary. Image processing can be applied in two ways: First, a color threshold is used to make a high contrast image where only information from one color range is extracted, then the pixels in the image are filled in to make shapes continuous using a reshape method. Hue, Saturation, and Luminance values can be adjusted to match the color information of the needle.

Set Image Processing.  Click to expand.

Set Image Processing.  Click to expand.

 

11.Alternately, these values can be programmatically extracted from the image. The steps below describe the Image Processing procedure:

a.Zoom in on the image and draw a rectangle inside the needle capturing the pixels of the desired color range.

b.Click 'Extract from image ROI' to fill in the values for Min and Max HSL from the selected ROI.

c.Zoom out to see the whole image and check the box next to 'Apply Color Threshold'.

d.Adjust Min and Max values until needle shows up on the image and background pixels are reduced. For some shades of Red, Hue Min and Max values should be inverted. (Check the 'invert Hue' box, and adjust Min and Max values again to compare the result)

e.Once color threshold is set, the image can be 'smoothed' and background pixels removed using reshape method. Check 'Reshape objects' and select the desired pixel radius and reshape method. By default, a 'square' radius around each pixel is processed. Check 'Hexagon' to use a hexagonal radius, which may improve the processing in some cases.

f.Once the processing is applied, make sure it works the same on all sample images. (The needle may have a different shade from different angles). A few background noise pixels will not prevent needle recognition as long as the needle has the shape of a line on the image.  

 

Programmatic Image Procesing.  Click to expand.

Programmatic Image Procesing.  Click to expand.

 

12.Set the needle minimum coordinates on the Test Pattern Match tab:

a.First, select the Needle Color (Light on Dark or Dark on Light).

b.Next, select the sample image that represents the needle at the minimum position. The green arrow should be placed on top of the needle, with tip and base of the line matching the contours of the needle on the image. Select the Line ROI tool from the toolbar under the image, and drag the line and line ends so it matches the image of the needle.
 
Note: Check the numeric coordinates to be sure the base and tip are not reversed.

Selecting needle coordinates.  Click to expand.

Selecting needle coordinates.  Click to expand.

 

13.Repeat the same for the sample image that shows the needle in the maximum position. First, select the image using the < > keys under the image. Then select the Line tool and drag the red arrow to place it on top of the needle at the maximum position.
 
If the needle rotates in a Counter Clockwise direction, put a check mark next to CCW rotation.

Selecting maximum needle coordinates.  Click to expand.

Selecting maximum needle coordinates.  Click to expand.

 

14.Now Needle can be recognized on the image. Press the Run Test button to test the algorithm. As long as the tip of the needle follows the arc from minimum position to maximum position, the algorithm will return the value where tip of the needle was found. Needle position (%) shows where the needle is in percentage from 0-100 of the total range.

15.Click the Signal Port Properties tab to specify the values to be returned during the test. Needle position (0-100%) is scaled based on the Min and Max values set here. The minimum position can be a negative value if the needle minimum position is not 0.

16.Enter a string to be used as engineering units for this signal.

Signal Port Properties.  Click to expand.

Signal Port Properties.  Click to expand.

 

17.Click OK. Now you can export transform ports and connect the SourceImage port to the SourceImage port on the camera transform.

Possible Problems

Moving Needle

The value of a moving needle is not meaningful. This problem is noticeable if the image of the needle is blurred. Design your test so that the tick period of the Camera Transform is synchronized with times when the needle is stationary.

Gauge Precision

Use Tolerances in your TestCases to allow for imprecision of the gauge.

Background Images

Images behind the range of needle motion, such as shown below, can cause problems.

NoiseImage

Possible solutions include the following:

Turn off the image if possible. In a lab situation, covering the image with black tape may be effective.

Crop the image out of the ROI.

It may be possible to eliminate the background image using Image Processing.

If the background image is the same color as the needle and the image cannot be eliminated, automated Image Recognition may not be possible.

Related Topics:

Recognition Modes

Camera Transform

Image Recognition Overview