ICV
Real-time computer vision for drilling rigs
ICV is a real-time computer vision system dvhb designed and built for an oilfield services company in 2020–2021. A server on the rig watches the drilling floor through the CCTV cameras, reads the rig’s sensors and warns the crew while an operation is going wrong, before a human error turns into an accident thousands of metres down.
- Client
- Oilfield services company, Europe
- Years
- 2020–2021
- Team
- 8 people at dvhb
- Stage
- Proof of concept on a working drilling rig
- Stack
- SSD object detection, optical flow, CVAT, RTSP, Modbus, MQTT and Protobuf, PostgreSQL, Grafana
The problem
Drilling crews follow strict procedures, and small slips are expensive. The client chose two to start with.
Drill string without movement
Left standing in the slips too long, the string can stick in the well, and freeing stuck pipe can stop drilling for days.
Casing make-up
Every casing connection needs the right number of turns. A bad one fails thousands of metres down, where it is very costly to fix.
An alert helps only while the crew can still act, so the system had to see and decide in real time, on the rig. Before writing code, the team went through the client’s incident database and interviewed drilling supervisors, drillers and assistant drillers about how these incidents happen.
Marker 1
Drill string without movement
Each rule ICV checks is a marker: a business rule over what the cameras and sensors see. For the first one the team trained an SSD object detector on four classes of 700 frames each: slips open, slips closed, slips closed and covered, and the spider. A region of interest and scene segmentation keep the model on the rotary table. A log analyzer turns frame-by-frame detections into time intervals, filters the noise and raises an alert when the string stands still longer than the rig’s limit, such as 3 or 5 minutes.



Marker 2
Counting casing turns
Casing pipes carry no mark that shows a full turn, so the team used the dirt: every pipe has its own pattern on its surface. ICV learns that pattern and counts a turn each time it comes round again, with tracking, image stabilisation, feature detection and optical flow following the pipe through the make-up.
Architecture
Every rig gets its own well-site server, so detection runs next to the cameras and keeps working when the internet doesn’t.

- Inputs
- PoE cameras record to a video recorder that streams over RTSP; rig sensors connect over Modbus.
- Well-site server
- A video grabber feeds the CV module: detection, plus post-processing that drops impossible combinations. A sensor grabber publishes sensor data over MQTT with Protobuf. The log analyzer combines both, and the well manager applies the rig’s settings and sends alerts.
- Rig floor
- Sound and light alarms for the crew, and a dashboard for the supervisor with every event and its video clip.
- Cloud
- Events from all rigs over MQTT into PostgreSQL, the managers’ dashboard, mobile apps, and notifications by SMS, email, push and webhooks.

New markers had to be possible without new code, so the team also designed a visual marker composer, where a rule is a graph of detections, sensor values and conditions.
Data pipeline
The rig cameras produced terabytes of raw CCTV footage. The team automated the path from raw video to labelling in CVAT, GPU training and testing; more GPU capacity cut a training run from 18 hours to 2. A video archive with search, filters, automatic tagging, clip cutting and thumbnails let the team find and label new situations fast.
Field lessons
- Image quality
- Steam, snow, mud on the lens and night shifts, all of which the training data had to cover.
- Connectivity
- Mobile internet with a monthly data cap and low bandwidth, so video stays on the rig and only events and short clips go to the cloud.
- Hardware
- Overheating, GPU driver failures and power cuts; remote monitoring and debugging tools mattered as much as the models.

Team
Eight people at dvhb: a product designer, a data scientist and ML engineer with a background in oil and gas, a principal backend architect for high-load computer vision and ML systems, backend and frontend engineers, a DevOps engineer and a data manager, led by Alex Samson as computer vision solution architect.
