Milestone: Our Silicon Valley Office

Our Silicon Valley branch is officially open:


Always Prepared For Third Party Defects

Our system uses hardware based third party video encoders, in both analog and IP cameras.
I have seen problems in both.

Analog encoder(Stretch S7) hangs occasionally. After days of research, I found it’s caused by a certain range of lighting, when it’s not daytime and not completely dark either.
The fix was to reboot the card when the hardware watch dog detects it.
IP camera (Vicon IQeye 9 series IQB92WI firmware V4.2) hangs ffmpeg HLS segmenter in about three hours, with error “Non-monotonous DTS”.
The fix was to downgrade to the previous version of firmware.

Those are both great products, but there are always corner cases.
A exhaustive test run is the only way to go. No reasoning is enough.

Video Player Selection

Video Player is a crucial part of our streaming system.
Both in house player and third-party players are used.
Some of our needs are not very common.
-Live video: Most players are designed for VOD
-Seeking on live streams
-Support mobile OS, such as Android and iOS
JW Player: the most popular web based player, proprietary.
version 8 can seek live stream on desktop, but not on mobile device.
Clappr Player can seek live stream in Chrome browser on Android
HLS.js with some customization, can seek live stream on Android when playing in web mode.
AVPlayer is used on iOS, on which our native app is based.

Bottom line: no single player can meet our needs on all three platforms: Windows, Android and iOS.

AI Player Recognition

The goal is to recognize hockey players by his uniform number (and then control the camera to follow him).
Great accuracy has been achieved in third quarter of 2017, thanks to our Chinese contractor, Zhao.

We went through two iterations, both involve OpenCV and TensorFlow.
Iteration 1
Number Bipmap Capture: OpenCV
Number Recognition: TensorFlow/Keras, CNN, MNIST dataset
-As the uniform number is usually two digits, CNN/MNIST only takes single digit bitmap as input.
Separating the digits is challenging.
-OpenCV captures all number without considering the context/background.
However, only numbers on uniform should be captured.
In order to conquer these two problems, we moved to iteration 2.
Iteration 2
Number Bipmap Capture: Tensorflow Object Detection Api(June 2017) replaces foreground detection.
By this technology, only numbers on a human are captured, regardless he’s moving or not.
Number Recognition:
SVHN replaces MNIST so that we can send a bitmap of “12” instead of sending two bitmaps for 1 and 2.