Day 2 ‐ Real‐Time Loop Counting in Knitting

Day 2 - Research and Model Building

Today, I dove into TensorFlow, MediaPipe, and OpenCV for various projects.

As humans, we make mistakes. It's frustrating to repeatedly fix miscounts or dropped rows. My aim is to accurately count the loops on each hand in real-time, watching for any dropped or added stitches.

To enhance accuracy and performance, I'm refining the data and developing a new model. I'm referencing knitting and crocheting training datasets on Kaggle:

• Crochet Samples V3
• UCF101 Action Recognition - Knitting Directory

Object Detection with TensorFlow:

My detection of the space between hand controllers wasn't as accurate as I'd hoped. Detecting gaps between needle colors isn't enough. If masking isn't precise, the model will need to learn to catch outliers and anomalies.

I wanted to count the loops highlighted in pink and remove the region highlight to simplify the count amidst other objects.

Here's what I tried:

1. First Attempt: Tracing the number of loops on each needle for both hands.

2. Second Attempt: Removing the green highlight to better detect hand positions between stitches and count stitches. Found the region but missed the count.

3. Third Attempt: Focusing on counting loops on a single needle between the hands by removing the region highlight.

4. Final Attempt: The final attempt to map the looms/loops per needle. That was inaccurate... as the counting was inconsistent, often missing or double-counting loops when observed.

I need to account for outliers, abnormalities, anomalies, and any other statistical variations that should be included in the tracing process.

Things i need to check:

1. Missing Data:

   • Solution: Implement a detection algorithm that identifies gaps and interpolates missing loops based on surrounding data.
   • User Prompt: "Please ensure both hands are fully visible. If loops are missing, try moving your hands closer to the camera."

2. Noise:

   • Solution: Use noise reduction techniques and filters to minimize random variations in detection.
   • User Prompt: "We detected some noise. Please hold your hands steady and ensure the background is clear of distractions."

3. Trends:

   • Solution: Analyze loop counts over multiple rows to identify and adjust for any trends.
   • User Prompt: "It looks like your loop count is more/less than what was initially done from the start. Please maintain a consistent knitting technique and hand position."

4. Bias:

   • Solution: Calibrate the detection system regularly to ensure it accurately counts loops without systematic errors.
   • User Prompt: "We detected a consistent bias. Please adjust your hand position and ensure the camera angle is optimal."

5. Purl Alignment:

   • Solution: Ensure the purl stitches are aligned correctly to improve loop detection.
   • User Prompt: "Please ensure your purl stitches are properly aligned to help the system accurately read the loops."

6. Data Distribution:

   • Solution: Analyze the distribution of loop counts to identify normal and abnormal patterns.
   • User Prompt: "We observed unusual loop counts. Please ensure both hands are clearly visible and centered in the frame."

General User Prompts for Better Tracking:

• "Ensure both hands are always visible and within the camera's field of view."
• "Move your hands closer to the camera if the loops are not being detected accurately."
• "Move your hands further away from the camera if the loops appear too large and are being miscounted."
• "Maintain a steady hand position and consistent knitting technique to improve detection accuracy."
• "Make sure the background is clear and well-lit to reduce detection errors."

Why Train a Model? Training a model to count loops between hands and needles is vital for consistency in knitting and crocheting, especially with circular needles where traditional counting methods might fail. 🧶

Steps to Create the Model using TensorFlow Model Builder:

1. Data Collection: Compile relevant datasets from Kaggle.
2. Data Preprocessing: Clean and format the data for training.
3. Model Training: Utilize TensorFlow Model Builder to train the model.
4. Evaluation: Assess the model's performance and make necessary adjustments.
5. Deployment: Deploy the model for real-time loop counting.

I'm hosting the dataset on Kaggle so it can be easily cited in Python. I'll ensure the model looks much better than I did in these photos (Excuse my eczema, patchy skin, and no make-up; just being transparent)! 😅

To be continued next week after spending time with my little one. 🐣

Additional Resources:

• Object Detection Counting with TensorFlow
• Masking Objects in Object Detection Counting
• Model Maker Documentation