Merge branch 'master' of https://github.com/UnavailableDev/avans-dui

3 files changed, 179 insertions, 0 deletions

diff --git a/assets/signs.png b/assets/signs.png
new file mode 100644
index 0000000..8102ac4
--- /dev/null
+++ b/assets/signs.png
diff --git a/doc/dui.md b/doc/dui.md
index 07c36c1..e4d14a3 100644
--- a/doc/dui.md
+++ b/doc/dui.md
@@ -383,10 +383,141 @@ combination with the standard Pololu boards and Libraries.
 }
 \buildSystemConclusion
 
+## Traffic Sign Detection (TSD)
+
+The following chapters will look at the different methods used to detect signs in order to find a suitable solution for this project.
+
+### Color based
+
+The distinct color characteristics of traffic signs can attract drivers’ attention and can also provide important cues to design color based detection methods. In the past decades, a large amount of detection methods are designed to detect distinct traffic sign colors such as blue, red and yellow. These methods can be directly used for traffic sign detection, and can also be used for preliminary reduction of the search space,  followed by other detection methods
+
+#### RGB
+
+One can easily look at the RGB values to detect a certain color. Although the r,g and b values are heavily effected by different illuminations, therefore this isn't a reliable solution in variating lighting conditions.
+
+An example implementation: 
+
+```py
+#Red
+if(R >= ThR and G <= thG)
+
+#Blue
+if(B >= thB)
+
+#Yellow
+if((R + G) >= ThY)
+```
+
+It is possible to enhance the colors with maximum and minimum operations:
+
+```py
+fR(i) = max(0, min(iR − iG, iR − iB)/s),
+fB(i) = max(0, min(iB − iR, iB − iG)/s),
+fY(i) = max(0, min(iR − iB, iG − iB)/s).
+```
+
+This method can result in some issues on the blue channel (see source 1 page 86583 for more explanation). As a solution to this issue use the following formula for the blue channel instead:
+```py
+f′B(i) = max((0, iB − iR)/s).
+```
+
+#### HSV
+
+The HSV/HSI color space is more immune to the illumination challenges of RGB. The hue and saturation channels can be calculated using RGB, which increases the processing time.
+The following pseudo code shows how to detect the red, blue and yellow colors in this space.
+
+```python
+#Red
+if (H <= Th1 or 
+    H >= Th2)
+
+#Blue
+if (H >= Th1 and 
+    H <= Th2)
+
+#Yellow
+
+if (H >= Th1 and 
+    H <= Th2 and 
+    H <= Th3)
+```
+
+#### LAB
+
+This color space is used for finding uncorrelated color components, the L\*a\*b\* space was used for detecting blue, red, yellow and green colors. 
+
+#### Pixel classification
+
+This method avoids the use of fixed thresholds that might need adjusting at times. In order to resolve this some authors tried to transfer the problem into pixel classification where a neural network classifies every pixel in the input image, the pixel classification algorithms are often slower than other color extraction methods.
+
+#### results
+
+\def\signDetectionColor{
+The above described methods where also applied to a database in order to compare each method. This resulted in the conclusion that, using a normalized RGB space is giving the a mixture of most detections and least false-positve results. See source 1 page 86584 for the full report.
+}
+\signDetectionColor
+
+### Shape based
+
+Common standard shapes of traffic signs are triangle, circle, rectangle, and octagon. Shape characteristics used for shape detection include standard shapes, boundaries, texture, key points, etc.
+
+#### Hough
+
+See line detection hough.
+<!-- TODO: not enough info found -->
+
+#### Barnes *Fang et al* (fast radial symmetry)
+
+Seemingly a 'simpler' method 
+<!-- TODO: (currently no great references found) -->
+
+#### Fourier
+This method also deals with occlusion and morphing/rotating the shape flat again (when looking at it at an angle).
+
+#### Key points detection
+This, simply put, looks at the edges/corners in order to find an ROI.
+
+### Color & Shape based
+In some methods, the shape detection methods can be combined with color based methods to fulfill the traffic sign detection work.
+For example, using a color based detection to find a ROI and using shape detection in that ROI to determine if it is a sign.
+
+### Neural networks
+<!-- TODO: -->
+
+### results
+
+\def\signDetectionShape{
+Most shape based recognition methods are more complex than using a color based detection. But the method 'Fourier' seems the most useful as it can also deal with rotated and occluded objects. 
+}
+\signDetectionShape
+
+## Traffic Sign Recognition (TSR)
+After traffic sign detection or tracking, traffic sign recognition is performed to classify the detected traffic signs into correct classes.
+
+![signs example](../assets/signs.png)
+
+### Binary tree
+The binary-tree-based classification method usually classify traffic signs according to the shapes and colors in a coarse-to-fine tree process.
+
+### Support Vector Machine (SVM)
+As a binary-classification method, SVM classifies traffic signs using one-vs-one or one-vs-others classification process.
+
+## results
+
+\def\signRecognition{
+While making a binary tree is seemingly the most simple, yet effective solution. Using the 'Fourier' method is a bit more complex, it may be a better solution (this requires testing).
+}
+\signRecognition
+
 # Conclusion
 
 \communicationConclusion
 \buildSystemConclusion
+\signDetectionColor
+\signDetectionShape
+\signRecognition
 
+## Sources:
 
+1. [IEEE, Digital Object Identifier June 26, 2019 (pages 86578 - 86596)](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8746141)
 
diff --git a/openMV/POC_signs_red.py b/openMV/POC_signs_red.py
new file mode 100644
index 0000000..cb521b9
--- /dev/null
+++ b/openMV/POC_signs_red.py
@@ -0,0 +1,48 @@
+# Hello World Example
+#
+# Welcome to the OpenMV IDE! Click on the green run arrow button below to run the script!
+
+import sensor, image, time
+
+# Color Tracking Thresholds (Grayscale Min, Grayscale Max)
+min_rgb = 128
+max_rgb = 255
+threshold_list = [(min_rgb, max_rgb)]# only bright grey colours will get tracked.
+threshold_rgb = [(0, 100, 75, 32, 2, 127)] #only find red
+#threshold_rgb = [(18, 78, -8, 127, 24, 127)]
+
+sensor.reset()                      # Reset and initialize the sensor.
+sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565 (or GRAYSCALE)
+#sensor.set_pixformat(sensor.GRAYSCALE)
+sensor.set_framesize(sensor.QVGA)   # Set frame size to QVGA (320x240)
+sensor.skip_frames(time = 2000)     # Wait for settings take effect.
+clock = time.clock()                # Create a clock object to track the FPS.
+
+while(True):
+    clock.tick()                    # Update the FPS clock.
+    img = sensor.snapshot()         # Take a picture and return the image.
+
+    #lines = img.find_lines()
+    #for i in lines:
+        #img.draw_line(i.line(), 255, 8)
+
+    #gray = img
+    #gray.to_grayscale()
+    #img.find_edges(0)
+
+
+
+    blobs = img.find_blobs(threshold_rgb)
+    #blobs.count()
+    #print(blobs)
+    ##kpts = img.find_keypoints()
+    for index, b in enumerate(blobs, 1):
+        convex = b.convexity()
+        if convex < 0.8:
+            img.draw_rectangle(b.rect(),int((512+256)*convex),2)
+        print(b.convexity())
+
+    #img.draw_line(12,12,200,200,255,8)
+
+    print(clock.fps())              # Note: OpenMV Cam runs about half as fast when connected
+                                    # to the IDE. The FPS should increase once disconnected.