Overview
The purpose of this project is detecting hidden shapes in the pictures then group them with their shapes and placements. Each group has a unique code, for example, if triangle shapes(SHAPE CODE: 3) with square placement(PLACEMENT CODE: 4), the code will be ‘34’.
Taking image input from the iPhone/Android Camera, then process the images with GPUImage/OpenCV libraries to visualize the images. To detect the shapes, train the visualized images with yolov8 ML model and converted it into a CoreML model to display recognition results on the iPhone, and train with Tensorflow Lite library(MobileNet V2 model) for the Android devices.
Languages and Libraries
- iOS App: Swift + Storyboard
- Android App: Kotlin
- Windows Desktop App: .NET 8 + WPF
- Machine Learning Model
- iOS: yolov8 model -> CoreML conversion
- Android: Tensorflow Lite -> Float16 Quantization
- GPUImage2 - iOS
- I tried GPUImage3(the latest version), but the library has changed its structure and some filters are not implemented(yet?). So, I had to GPUImage2 and added custom filter for barcode detector.
- Android-GPUImage - Android
- OpenGL ES 2.0 Based Filters
- OpenCV 3.4.x - iOS/Android
- I also tried OpenCV 4.x, but I couldn’t get OpenCV 4.x Camera input on Android Devices correctly(not sure what caused it), so I tested V3.4.16 and chose it.