Craft beer vision scanner
A concept for recognising barcodeless craft cans at checkout: what the problem is, how the solution would work, and how the pieces fit together.
The problem
This is something I have experienced almost every time I buy the beers I enjoy at a bottle store. No barcodes, or barcodes reused across products, mean that delays are inevitable, as cashiers search for the right keyword, are often unable to find it, and end up calling in assistance from others to try to determine what keyword the beer was saved under.
Constant new releases are what makes a craft brewery a proper craft brewery. They're also what makes these beers hard to ring up.
1.1How shops cope today
Search by keywords
- How it works
- Staff key each beer into the till under a brewery, beer name or short form. Cashiers search for it.
- Where it breaks
- Relies on staff experience and knowing the range.
- Keywords aren't obvious, especially for brewery collabs etc.
- Can designs often don't make clear which text is the beer, the brewery or the style – not all cashiers are craft beer experts.
- Different people save things under different names.
The shop's own barcode stickers
- How it works
- The shop prints its own barcodes and sticks one on every can, so it scans like any other product.
- Where it breaks
- Manual work on every single can.
- Sticker and printer stock to keep up.
- Condensation on cold, wet cans: stickers fall off or crumple.
1.2The new idea: Craft Beer Vision Scanner
A phone on a small stand (end hardware design tbc) at the till recognises the can and hands the till the shop's own code for that beer, exactly as a barcode scanner would. The cashier checks the name, price and pack size before anything is sent.
Bolts on, replaces nothing
To the till it looks like an ordinary barcode scanner. That means it should work with any POS capable of third-party hardware integration (almost all POS have this).
The cashier always decides
Final confirmation for the cashier – if the tolerance level is below a pre-determined configurable % it will show more than 1 option (in order of decreasing likelihood) and the cashier picks.
Fixes the names at the source
When stock arrives, the can is scanned once. The app reads it and files one clean record with every name/combination it might be searched by (i.e. it will also cater for manual interventions using keyword search).
The exact hardware and software are still open. A short proof of concept decides them, using real cans and a real till (see 2.5).
Architecture
2.1How the pieces fit together
Everything time-critical happens on the phone/device at the counter, so a scan doesn't wait on the internet. The cloud holds the shared catalogue, reads new cans when stock arrives, and keeps a log of every confirmed scan.
2.2At the till: the checkout scan
The name on the can is the most precise clue, so it's trusted first. The can's design is the fallback for when the text is stylised, glared or a confusing collab. Both checks start the moment the can is scanned, so the cashier only sees the answer.
Collabs match as a set of breweries in any order, so OC × ACBC and ACBC × OC are the same beer. Each confirmed scan can also add its photo as a further reference, so picture matching improves with use.
Intake of new stock can also (configurable) pre-populate suggested keywords to help cashiers if manual intervention is needed.
2.3When new stock arrives
The goal here is to also remove keyword search issues for the current manual path. It takes a little longer than an instant checkout scan (a couple of seconds is fine here) because it uses the cloud AI read, but is only done once per product.
Photograph the can
The front, plus an optional turn to catch collab names or small print.
AI reads it
Every brewery including collab partners ("x", "&", "collab with"), beer name, style, ABV and size, each with a confidence level.
AI can-reading functionTidy the names
Breweries are matched against known aliases. A new brewery gets suggested short forms for a person to approve.
Already on file?
The photo runs through the checkout match. If the beer is known, one tap adds the photo to improve matching.
Find its till code
Searches the till's item list by every name and alias and offers the top 3. If it isn't in the till yet, suggests a consistent name for staff to key in.
Link pack sizes
Single, 4-pack and 6-pack codes, each with its price.
Save
One clean record, plus its first reference photo and fingerprint.
MediaPipe · Supabase
- Breweries
- OC Brewery OC × Afro Caribbean Brewing Company ACBCAfro
- Beer
- Mango Sour · sour · 330 ml
- Till's old name
- ACBC/OC MANGO SOUR 330 kept as a search key, not thrown away
- Till codes
- Single 4471 · 4-pack 4472, each with its price
- Found by
- afroACBCOCmango sourACBC/OC or the camera
- Also kept
- ABV, short description and a clean front photo, plus reference photos for matching
To get started, the till's existing item list is imported once and the AI proposes clean names for a person to approve. The same flow in bulk mode photographs the range already on the shelf.
2.4The components, step by step
| Step | What happens | Component | Runs on | |
|---|---|---|---|---|
| 1 | Read the name | On-device text reading pulls the brewery and beer name off the can and matches it against every name and alias. | Google ML Kit Text Recognition | Phone, offline |
| 2 | Match the design | If the text is ambiguous, the image is compared with the shop's reference photos, by converting the image data into a high-dimensional numeric representation.(Machine learning, not a large AI model, so it can run on-device) | MediaPipe Image Embedder | Phone, offline |
| 3 | Check tolerance | Configurable thresholds decide whether a match is clear enough, and how far ahead of the runner-up it must be. | Per-shop settings | Phone |
| 4 | Quick confirm | The cashier sees name, price and pack and taps Confirm, or picks from up to 3 candidates. | Android app (Kotlin, Jetpack Compose) | Phone |
| 5 | Send to the till | The till code is typed into the till followed by Enter, as if it had been scanned. The code is also always shown on screen. | USB bridge (ESP32-S3) or Bluetooth keyboard mode | Till |
| + | Read new cans | Reads breweries, collab partners, beer name, style, ABV and size when stock arrives. Backup at checkout. | Supabase Edge Function + Gemini Flash-Lite or Claude Haiku | Cloud |
| + | Keep the catalogue | Breweries, beers, aliases, till codes, reference photos and the scan log. | Supabase (Postgres + storage) | Cloud, EU |
| + | Manage it | Import the item list, approve names, manage aliases, review the scan log. | Single web page on Cloudflare Pages | Browser |
2.5Still to prove
The order of the checks is fixed. Which parts do each job is decided by testing with real cans from the shop and a real till.
| Open question | Options | How it gets decided |
|---|---|---|
| Is reading the name on the phone good enough as the first check? | ML Kit on its own, or the cloud AI read used more often | Bake-off on 25–30 real cans, including collabs and lookalikes |
| Picture matching on the phone or in the cloud? | MediaPipe on the phone, or a hosted model (Vertex multimodal / DINOv2) | Same bake-off |
| Which AI reads cans best? | Gemini Flash-Lite or Claude Haiku | Same bake-off, scored against the shop's real till names |
| How does the code reach the till? | USB bridge, or the phone as a Bluetooth keyboard | A 15-minute test at a quiet till |
| Phone on a stand, handheld, or another device? | Stand with tap or auto-capture; handheld with a volume-key shutter for wet hands | Test different options and decide |
Talk through how the till, naming and new stock work today.
Laptop bake-off on real cans against the shop's real till names.
A simple app that shows the right code on screen, timed against how cashiers find beers now.
Where it could go
The scanner is the first step. Once every craft beer has one clean record, other things become possible. These are ideas for later, not plans.
Delivery listings without the retyping
Each record already holds the style, ABV, description and a clean photo. The same data could produce ready-to-upload Uber Eats and Mr D listings for beers with no barcode, which today have to be created by hand one at a time. Potential for automated listings depending on brewery collaboration (see next idea).
Breweries send their label once
A shared master library of label artwork, supplied by breweries at or before release, allowing pre-rendered photorealistic can image generation, so a new beer is recognised at every till on day one and nobody has to photograph it at the input stage. (Feeds into delivery app listings tool and future ideas...)
The bigger aim
- Better experiences for customers
- More reach for South Africa's craft breweries, with less manual work / cost / friction standing in their way to finding new audiences and customers.
- Ultimately, I would like to help to steer the SA craft scene in general to get closer to what I've experienced in other markets like the UK, the US and Australia.
This is one idea of several. Let's talk it through, and hear what else gets in the way, over a beer.
Backyard Singularity: Embrace the chaos.