Infrastructure
To support TNO it requires several distributed environments and different types of architecture. Within government there is the local office, on-premise infrastructure, and on-premise Openshift infrastructure. Externally cloud based infrastructure is hosted within Azure and GitHub.
The TNO solution environments have not yet been fully designed or implemented.
The following diagram provides some insight into how the on location office is setup to support the capture of audio, video, and online data sources.
Diagram details not yet complete
All inputs to the NAS are through network connection. Kalel plays a significant role in all channel selections. Please check under Kalel>File>Events for text selection commands.
- (channel selected at CC tuner)
- Input - AC & radio signals Output 3.5mm (F) to Linux PC Fedora 10 - Delta 1010LT PCI card
- (channel selected at Grace tuner)
- Input - network connection & AC Output - two RCA (F) to Linux PC Fedora 10 - Delta 1010LT PCI card
- (channels are selected on the PC and in Kalel)
- Input - AC & coax cable feed from Legislature passing through numerous amps Output - coax (F) analogue Victoria Shaw television feed to multiple splitters to Linux PC Fedora 10 - WinTV-HVR-1600 card
- (channels selected at the Shaw tuner)
- Input - AC, satellite dish (roof 617 Gov't) - splitter - Shaw tuners Output - coax (F) to Linux PC Fedora 10 - WinTV-HVR-1600 card
- (channels selected at the Shaw tuner)
- Input - AC, Shaw coax feed - Shaw tuners Output - coax (F) to Linux PC Fedora 10 - WinTV-HVR-1600 card
- (channel selected at Shaw tuner)
- Input - network, AC & USB 2.0 Output - Network connection to NAS Connection
- note - HDMI output from Shaw tuner to HDMI splitter to HDMI to USB converter to USB input in Raspberry PI. (the HDMI splitter was inserted to overcome propriety restrictions)
The support budget for this equipment is included in the AKTIV support contract which includes application support, 24/7 on call support and all troubleshooting work. All of the current A/V capture solutions were created using off-the-shelf hardware. GDX is interested in bringing all the support for TNO in-house. As a result, we are looking for a more reliable, purpose built solution for all AV capturing. The current system requires daily on-site maintenance and would not meet the WFH requirement of the current upgrade.
Please note, many of these solutions are expiring. We had to move to Raspberry PI as the new WinTV cards and drivers had computability issues. Similarly, the Delta cards no longer work in the gov't PCs we can acquire.
Audio and Video streams are collected using a variety of devices but in each case the process follows these steps:
- The audio/video signal is received by a tuner or digital capture device.
- The capture device feeds a signal into an A/V card mounted in a Lenovo PC or Raspberry PI (running Linux).
- The A/V card maps to a device in the host operating system's /dev directory.
- Once per day Jorel spawns an ffmpeg capture process that streams the digital content from this device and stores it in a single file in local storage.
- At intervals, defined in the Jorel event table, clips are extracted from the parent file and are stored on the NAS located in Victoria.
- The timespan of these clips overlap so that there is no missed content.
- As the clips are extracted based on the time of day, it is important that, should the capture event start late, the start time of recording is noted in the database. This then becomes the timestamp relative to which the clips are extracted.
- Relevant clips are imported into QuickTime by editors, where the overlapping sections and any unwanted content are removed.
- The curated clips are then saved to local storage on the editor's workstation.
- The editors use Kalel to upload the completed clips to the web-accessible storage in Kamloops and create a corresponding news item.
- At the end of the day a ShellCommand event runs that terminates the capture process.
The Events table contains records describing 18 media servers that appear to be active. A media server can capture at most two television channels or six radio channels. Some servers support a mix of TV and radio capture, in which case at most one TV channel can be processed. The table below shows the channel assignments of the various servers:
| Server Process | Sources | Type |
|---|---|---|
| media1 | CIVT, BNN | TV News |
| media2 | CBYK, CFAX, CKNW, CKFU, CBU_2, CJCN | Talk Radio |
| media3 | CKPG, CBUT | TV News |
| media4 | CFJC, CHAN | TV News |
| media5 | CHBC | TV News |
| media6 | CIVI, Global | TV News |
| media7 | CHECK, CTV | TV News |
| media8 | CJRJ, CHNM TV, CKSP, CKYE | TV News/Talk Radio |
| media9 | CHNL, CBYG, CBU, CBCV, CKFR, CKWX | Talk Radio |
| media10 | CHMB, CBTK, CJVB, CACA, NWTEST | Talk Radio |
| media11X | PTC | TV News |
| mediaHD2 | CFTV | TV News |
| mediaHD2X | ZeeTV | TV News |
| mediaHD3X | HDCHAN6 | TV News |
| mediaHDRasp1 | CBC Newsworld | TV News |
| mediaHDRasp2 | CBUT | TV News |
| mediaHDRasp3 | CKPG, APTN | TV News |
The capture commands are run by Jorel either at 12:10 am or 3:00 am every day to start capturing a day’s worth of programming. The values in square brackets, in the commands below, are substitution variables that are replaced with values from other fields in related database tables or files. Each capture command has a set of associated clip commands that are executed throughout the day. Analog video Capture events may also have a related ccCapture event that manages the extraction of closed-captioned text from the video stream. Some of the data from the Capture, Clip and (for analog video only) ccCapture events are shown below. (ffmpeg is a command-line utility that provides fine-grained video processing capabilities.)
| CHMB Talk Radio | |
|---|---|
| Capture | ffmpeg -f oss -i /dev/[channel] -t [duration] -acodec mp2 -ab 64k -ar 22050 -vol 550 -y [capture] &> /dev/null |
| Clip | ffmpeg -ss [start] -i [capture] -t [duration] -f mov -acodec alac -ab 64k -ar 22050 -vol 550 -y [clip].mov &> /dev/null |
| Global TV News | |
|---|---|
| Capture | cat /dev/[channel] | tee [capture] | ffmpeg -i - -t [duration] -f mpeg -vcodec mpeg2video -b 1000k -s 360x240 -r 25 -acodec mp2 -ab 128k -y [streamlocal].mpg &> /dev/null |
| Clip | ffmpeg -ss [start] -i [streamlocal].mpg -t [duration] -f mov -vcodec mpeg4 -sameq -vsync 1 -acodec alac -ab 128k -async 22050 -vol 75 -y [clip].mov &> /dev/null |
| ccCapture | ccextractor -i [input].cc -startat [start] -endat [stop] -noru -delay [delay] -out=txt -sc -o [tempfile][ext] &> /dev/null; /jorel/fix_cc.sh [tempfile][ext] [output][ext] &> /dev/null |
Clip commands are also run by Jorel on a fixed schedule and extract a portion of the captured video stream from a start time to an end time. The clip values are templates with substitution variables that are populated from the EVENT_CLIPS table. Clips run many times per day and slice the file being captured into usable chunks.
If the CC_CAPTURE column of the Capture event record is set to true, related clip events will be provided in pairs with shared start and end times. One event will extract the av clip, the other will extract the text. The closed-captioned text is extracted from the captured video stream, using the ccextractor command line utility, and stored in the clips directory with a file name that matches that of the clip. The closed-captioned text is not included in files captured from a high-definition digital source.
Here are the EVENT_CLIPS records for CHMB and Global TV News. There's a lot of information in the following two tables, but they give a sense of the granularity of the audio/video clipping process.
| RSN | EVENT_RSN | NAME | START_TIME | STOP_TIME | FREQUENCY | LAST_RUN |
|---|---|---|---|---|---|---|
| 144817037 | 144817017 | 0500 | 04:58 | 05:20 | mtwtfss | Nov 17 2021 |
| 153158345 | 144817017 | 0600 | 05:58 | 06:10 | mtwtfss | Nov 17 2021 |
| 144817040 | 144817017 | 0600a | 05:58 | 06:20 | mtwtfss | Nov 17 2021 |
| 144817050 | 144817017 | 0700a | 06:58 | 07:20 | mtwtfss | Nov 17 2021 |
| 153158369 | 144817017 | 0800 | 07:58 | 08:30 | mtwtfss | Nov 17 2021 |
| 144817051 | 144817017 | 0800a | 07:58 | 08:30 | mtwtfss | Nov 17 2021 |
| 153158400 | 144817017 | 0900 | 08:58 | 09:10 | mtwtfss | Nov 17 2021 |
| 144817054 | 144817017 | 0900a | 08:58 | 09:20 | mtwtfss | Nov 17 2021 |
| 153158406 | 144817017 | 1000 | 09:58 | 10:10 | mtwtfss | Nov 17 2021 |
| 144817059 | 144817017 | 1000a | 09:58 | 10:20 | mtwtfss | Nov 17 2021 |
| 153158351 | 144817017 | Sat | 10:55 | 12:10 | mtwtfss | Nov 16 2021 |
| 144818655 | 144817017 | 1100a | 10:58 | 11:20 | mtwtfss | Nov 17 2021 |
| 153158411 | 144817017 | 1100 | 10:58 | 11:10 | mtwtfss | Nov 17 2021 |
| 425995830 | 144817017 | 1200z | 11:58 | 13:01 | mtwtfss | Nov 16 2021 |
| 144817033 | 144817017 | 1200a | 11:58 | 12:35 | mtwtfss | Nov 16 2021 |
| 153158415 | 144817017 | 1200 | 11:58 | 12:10 | mtwtfss | Nov 16 2021 |
| 144822501 | 144817017 | 1230 | 12:28 | 12:40 | mtwtfss | Nov 16 2021 |
| 144817034 | 144817017 | 1300a | 12:58 | 13:20 | mtwtfss | Nov 16 2021 |
| 153158420 | 144817017 | 1300 | 12:58 | 13:10 | mtwtfss | Nov 16 2021 |
| 144817066 | 144817017 | 1400a | 13:58 | 14:20 | mtwtfss | Nov 16 2021 |
| 153158424 | 144817017 | 1400 | 13:58 | 14:10 | mtwtfss | Nov 16 2021 |
| 153158428 | 144817017 | 1500 | 14:58 | 15:10 | mtwtfss | Nov 16 2021 |
| 144817069 | 144817017 | 1500a | 14:58 | 15:20 | mtwtfss | Nov 16 2021 |
| 144817070 | 144817017 | 1600 | 15:58 | 16:10 | mtwtfss | Nov 16 2021 |
| 153158432 | 144817017 | 1600a | 15:58 | 16:15 | mtwtfss | Nov 16 2021 |
| 153846323 | 144817017 | 1600b | 15:58 | 16:20 | mtwtfss | Nov 16 2021 |
| 153846335 | 144817017 | 1600c | 15:58 | 16:30 | mtwtfss | Nov 16 2021 |
| 559727729 | 144817017 | For Cindya | 16:14 | 18:29 | mtwtf-- | Nov 16 2021 |
| 494486099 | 144817017 | For Cindy | 16:15 | 18:30 | mtwtf-- | Nov 16 2021 |
| 697558250 | 144817017 | Fri | 16:25 | 18:05 | mtwtf-- | Nov 16 2021 |
| 154119920 | 144817017 | 22a | 21:58:30 | 22:02 | mtwtfss | Nov 16 2021 |
| RSN | EVENT_RSN | NAME | START_TIME | STOP_TIME | FREQUENCY | LAST_RUN |
|---|---|---|---|---|---|---|
| 132180052 | 91572573 | 0410-0420 | 04:09 | 04:21 | mtwtfss | Nov 17 2021 |
| 132180053 | 91572573 | 0420-0430 | 04:19 | 04:31 | mtwtfss | Nov 17 2021 |
| 132180051 | 91572573 | 0430-0440 | 04:29 | 04:41 | mtwtfss | Nov 17 2021 |
| 132180050 | 91572573 | 0500-0510 | 05:00 | 05:10 | mtwtfss | Nov 17 2021 |
| 132180049 | 91572573 | 0520-0530 | 05:20 | 05:30 | mtwtfss | Nov 17 2021 |
| 681577797 | 91572573 | 0540-0550 | 05:28 | 05:58 | mtwtfss | Nov 17 2021 |
| 150373783 | 91572573 | 0540-0550 | 05:28 | 05:58 | mtwtfss | Nov 17 2021 |
| 91586386 | 91572573 | 0600-0611 | 05:58 | 06:11:00 | mtwtfss | Nov 17 2021 |
| 91586387 | 91572573 | 0610-0621 | 06:10:00 | 06:21:00 | mtwtfss | Nov 17 2021 |
| 91586388 | 91572573 | 0620-0631 | 06:20:00 | 06:31:00 | mtwtfss | Nov 17 2021 |
| 91586389 | 91572573 | 0630-0641 | 06:30:00 | 06:41:00 | mtwtfss | Nov 17 2021 |
| 91586390 | 91572573 | 0640-0651 | 06:40:00 | 06:51:00 | mtwtfss | Nov 17 2021 |
| 91586391 | 91572573 | 0750-0800 | 07:51 | 08:01 | mtwtfss | Nov 17 2021 |
| 91586392 | 91572573 | 0800-0810 | 07:59 | 08:11 | mtwtfss | Nov 17 2021 |
| 133941584 | 91572573 | 0810 - 0820 | 08:09 | 08:21 | mtwtfss | Nov 17 2021 |
| 133941601 | 91572573 | 0820-0830 | 08:19 | 08:31 | mtwtfss | Nov 17 2021 |
| 133941605 | 91572573 | 0830-0840 | 08:29 | 08:41 | mtwtfss | Nov 17 2021 |
| 133941613 | 91572573 | 0840-0850 | 08:39 | 08:51 | mtwtfss | Nov 17 2021 |
| 133941614 | 91572573 | 0850-0900 | 08:49 | 09:01 | mtwtfss | Nov 17 2021 |
| 132750404 | 91572573 | 0900-0910 | 08:58 | 09:11 | mtwtfss | Nov 17 2021 |
| 132871544 | 91572573 | 0910-0920 | 09:09 | 09:21 | mtwtfss | Nov 17 2021 |
| 132871551 | 91572573 | 0920-0930 | 09:19 | 09:31 | mtwtfss | Nov 17 2021 |
| 133040921 | 91572573 | 0930-0940 | 09:29 | 09:41 | mtwtfss | Nov 17 2021 |
| 133040933 | 91572573 | 0940-0950 | 09:39 | 09:51 | mtwtfss | Nov 17 2021 |
| 103301559 | 91572573 | 0950-1001 | 09:50:00 | 10:01:00 | mtwtfss | Nov 17 2021 |
| 133415293 | 91572573 | 0958-1009 | 09:58 | 10:09 | mtwtf-- | Nov 17 2021 |
| 103301560 | 91572573 | 1000-1011 | 10:00:00 | 10:11:00 | mtwtfss | Nov 17 2021 |
| 156289050 | 91572573 | 1000 - 1030 | 10:08 | 10:35 | mtwtfss | Nov 17 2021 |
| 103301564 | 91572573 | 1010-1021 | 10:10:00 | 10:21:00 | mtwtfss | Nov 17 2021 |
| 103301565 | 91572573 | 1020-1031 | 10:20:00 | 10:31:00 | mtwtfss | Nov 17 2021 |
| 103301579 | 91572573 | 1030-1041 | 10:30:00 | 10:41:00 | mtwtfss | Nov 17 2021 |
| 103301587 | 91572573 | 1040-1051 | 10:40:00 | 10:51:00 | mtwtfss | Nov 17 2021 |
| 103301590 | 91572573 | 1050-1101 | 10:50:00 | 11:01:00 | mtwtfss | Nov 17 2021 |
| 91586416 | 91572573 | 1100-1111 | 10:59 | 11:11:00 | mtwtfss | Nov 17 2021 |
| 91586417 | 91572573 | 1110-1121 | 11:10:00 | 11:21:00 | mtwtfss | Nov 17 2021 |
| 91586418 | 91572573 | 1120-1131 | 11:20:00 | 11:31:00 | mtwtfss | Nov 17 2021 |
| 91586419 | 91572573 | 1130-1141 | 11:30:00 | 11:41:00 | mtwtfss | Nov 16 2021 |
| 91586420 | 91572573 | 1140-1151 | 11:40:00 | 11:51:00 | mtwtfss | Nov 16 2021 |
| 91586421 | 91572573 | 1150-1201 | 11:50:00 | 12:01:00 | mtwtfss | Nov 16 2021 |
| 91586422 | 91572573 | 1200-1211 | 12:00:00 | 12:11:00 | mtwtfss | Nov 16 2021 |
| 91586423 | 91572573 | 1210-1221 | 12:05:00 | 12:21:00 | mtwtfss | Nov 16 2021 |
| 91586424 | 91572573 | 1220-1231 | 12:20:00 | 12:31:00 | mtwtfss | Nov 16 2021 |
| 551554197 | 91572573 | 1230-1241 | 12:30 | 12:41 | mtwtfss | Nov 16 2021 |
| 91586425 | 91572573 | 1240-1251 | 12:40 | 12:51 | mtwtfss | Nov 16 2021 |
| 103792937 | 91572573 | 1250-1301 | 12:49 | 13:01:00 | mtwtfss | Nov 16 2021 |
| 132513009 | 91572573 | 1300-1310 | 12:59 | 13:11:00 | mtwtfss | Nov 16 2021 |
| 132513026 | 91572573 | 1310-1320 | 13:09 | 13:21:00 | mtwtfss | Nov 16 2021 |
| 132513027 | 91572573 | 1320-1330 | 13:19 | 13:31:00 | mtwtfss | Nov 16 2021 |
| 132513036 | 91572573 | 1330-1340 | 13:29 | 13:41:00 | mtwtfss | Nov 16 2021 |
| 132284596 | 91572573 | 1340-1350 | 13:39 | 13:51 | mtwtfss | Nov 16 2021 |
| 132284614 | 91572573 | 1350-1400 | 13:49 | 14:01 | mtwtfss | Nov 16 2021 |
| 341969422 | 91572573 | 1355-1407 | 13:58 | 14:08 | mtwtfss | Nov 16 2021 |
| 132284684 | 91572573 | 1400-1410 | 13:59 | 14:11 | mtwtfss | Nov 16 2021 |
| 132284691 | 91572573 | 1410-1420 | 14:09 | 14:21 | mtwtfss | Nov 16 2021 |
| 132284698 | 91572573 | 1420-1430 | 14:19 | 14:31 | mtwtfss | Nov 16 2021 |
| 132401338 | 91572573 | 1430-1440 | 14:29 | 14:41 | mtwtfss | Nov 16 2021 |
| 132401340 | 91572573 | 1440-1450 | 14:39 | 14:51 | mtwtfss | Nov 16 2021 |
| 132401345 | 91572573 | 1450-1500 | 14:49 | 15:01 | mtwtfss | Nov 16 2021 |
| 614353095 | 91572573 | Dix/Henry | 14:55 | 15:58 | mtwtfs- | Nov 16 2021 |
| 132401346 | 91572573 | 1500 - 1510 | 14:59 | 15:11 | mtwtfss | Nov 16 2021 |
| 132401353 | 91572573 | 1510 - 1520 | 15:09 | 15:21 | mtwtfss | Nov 16 2021 |
| 132401354 | 91572573 | 1520 - 1530 | 15:19 | 15:31 | mtwtfss | Nov 16 2021 |
| 132401359 | 91572573 | 1530 - 1540 | 15:29 | 15:41 | mtwtfss | Nov 16 2021 |
| 132401388 | 91572573 | 1540 - 1550 | 15:39 | 15:51 | mtwtfss | Nov 16 2021 |
| 132401390 | 91572573 | 1550 - 1600 | 15:49 | 16:05 | mtwtfss | Nov 16 2021 |
| 647483487 | 91572573 | 1550 to 1558 | 15:50:00 | 15:58:00 | mtwtfss | Nov 16 2021 |
| 91586446 | 91572573 | 1600-1611 | 15:59 | 16:11:00 | mtwtfss | Nov 16 2021 |
| 91586447 | 91572573 | 1610-1621 | 16:09 | 16:21:00 | mtwtfss | Nov 16 2021 |
| 91586448 | 91572573 | 1620-1631 | 16:19 | 16:31:00 | mtwtfss | Nov 16 2021 |
| 91586449 | 91572573 | 1630-1641 | 16:29 | 16:41:00 | mtwtfss | Nov 16 2021 |
| 91586450 | 91572573 | 1640-1651 | 16:39 | 16:51:00 | mtwtfss | Nov 16 2021 |
| 91586451 | 91572573 | 1650-1701 | 16:49 | 17:01:00 | mtwtfss | Nov 16 2021 |
| 182965611 | 91572573 | CHAN1700_Overview | 16:50:00 | 17:20:00 | mtwtfss | Nov 16 2021 |
| 79215130 | 91572573 | 1700-1711 | 16:58 | 17:12:00 | mtwtfss | Nov 16 2021 |
| 91586453 | 91572573 | 1710-1721 | 17:09:00 | 17:21:00 | mtwtfss | Nov 16 2021 |
| 91586454 | 91572573 | 1720-1731 | 17:20:00 | 17:31:00 | mtwtfss | Nov 16 2021 |
| 91586455 | 91572573 | 1730-1741 | 17:30:00 | 17:41:00 | mtwtfss | Nov 16 2021 |
| 91586456 | 91572573 | 1740-1751 | 17:40:00 | 17:51:00 | mtwtfss | Nov 16 2021 |
| 91586457 | 91572573 | 1750-1801 | 17:50:00 | 18:01:00 | mtwtfss | Nov 16 2021 |
| 182965619 | 91572573 | CHAN1800_Overview | 17:50:00 | 18:45:00 | mtwtfss | Nov 16 2021 |
| 91586458 | 91572573 | 1800-1811 | 18:00:00 | 18:11:00 | mtwtfss | Nov 16 2021 |
| 91586459 | 91572573 | 1810-1821 | 18:10:00 | 18:21:00 | mtwtfss | Nov 16 2021 |
| 91586460 | 91572573 | 1820-1831 | 18:20:00 | 18:31:00 | mtwtfss | Nov 16 2021 |
| 91586461 | 91572573 | 1830-1841 | 18:30:00 | 18:41:00 | mtwtfss | Nov 16 2021 |
| 91586462 | 91572573 | 1840-1851 | 18:40:00 | 18:51:00 | mtwtfss | Nov 16 2021 |
| 91586463 | 91572573 | 1850-1901 | 18:50:00 | 19:03:00 | mtwtfss | Nov 16 2021 |
| 133129560 | 91572573 | 1900-1910 | 18:59 | 19:11 | mtwtfss | Nov 16 2021 |
| 133129564 | 91572573 | 1910-1920 | 19:09 | 19:21 | mtwtfss | Nov 16 2021 |
| 133129571 | 91572573 | 1920-1930 | 19:19 | 19:31 | mtwtfss | Nov 16 2021 |
| 133129576 | 91572573 | 1930-1940 | 19:29 | 19:41 | mtwtfss | Nov 16 2021 |
| 133129579 | 91572573 | 1930-1940 | 19:29 | 19:41 | mtwtfss | Nov 16 2021 |
| 133129596 | 91572573 | 1940-1950 | 19:39 | 19:51 | mtwtfss | Nov 16 2021 |
| 133129614 | 91572573 | 1950-2000 | 19:49 | 20:01 | mtwtfss | Nov 16 2021 |
| 349745706 | 91572573 | temp | 19:57:00 | 20:11:00 | mtwtfss | Nov 16 2021 |
| 133363227 | 91572573 | 2000-2020 | 19:58 | 20:22 | mtwtfss | Nov 16 2021 |
| 134989085 | 91572573 | TEST | 20:10 | 20:25 | mtwtfss | Nov 16 2021 |
| 133363229 | 91572573 | 2020-2040 | 20:18 | 20:42 | mtwtfss | Nov 16 2021 |
| 133363230 | 91572573 | 2040-2100 | 20:38 | 21:02 | mtwtfss | Nov 16 2021 |
| 133363237 | 91572573 | 2100-2120 | 20:58 | 21:22 | mtwtfss | Nov 16 2021 |
| 133363239 | 91572573 | 2120-2140 | 21:18 | 21:42 | mtwtfss | Nov 16 2021 |
| 133363247 | 91572573 | 2140-2200 | 21:38 | 22:02 | mtwtfss | Nov 16 2021 |
| 91586486 | 91572573 | 2200-2211 | 21:58:00 | 22:11:00 | mtwtfss | Nov 16 2021 |
| 91586487 | 91572573 | 2210-2221 | 22:10:00 | 22:21:00 | mtwtfss | Nov 16 2021 |
| 91586488 | 91572573 | 2220-2231 | 22:20:00 | 22:31:00 | mtwtfss | Nov 16 2021 |
| 91586489 | 91572573 | 2230-2241 | 22:30:00 | 22:41:00 | mtwtfss | Nov 16 2021 |
| 91586490 | 91572573 | 2240-2251 | 22:40:00 | 22:51:00 | mtwtfss | Nov 16 2021 |
| 91586491 | 91572573 | 2250-2301 | 22:50:00 | 23:01:00 | mtwtfss | Nov 16 2021 |
| 91586492 | 91572573 | 2300-2311 | 23:00:00 | 23:11:00 | mtwtfss | Nov 16 2021 |
| 91586493 | 91572573 | 2310-2321 | 23:10:00 | 23:21:00 | mtwtfss | Nov 16 2021 |
| 91586494 | 91572573 | 2320-2331 | 23:20:00 | 23:31:00 | mtwtfss | Nov 16 2021 |
| 91586495 | 91572573 | 2330-2341 | 23:30:00 | 23:41:00 | mtwtfss | Nov 16 2021 |
| 91586496 | 91572573 | 2340-2351 | 23:40:00 | 23:51:00 | mtwtfss | Nov 16 2021 |
| 134160259 | 91572573 | 2400-2415 | 24:01 | 24:20 | mtwtfss | ? |
Video and audio are captured using two different platforms:
- Lenovo tower systems running Linux (analog).
- Raspberry PI systems running Linux (digital).
The row of tower systems below left represents the majority of the media boxes listed in the events table. The two Raspberry PI devices are mediaHDRasp3 (right) mediaHDRasp2 (left). (Hovering your mouse over these images provides a description.)
The media boxes are arranged in ascending order, by name, from left to right. The image on the left includes media4, media5 and media6. These machines store the captured video on one of the 4 Terrabyte storage devices on the right (local NAS). The other NAS is spare and has been configured for use as a mapped drive for testing from home office locations.
Audio and Video signals enter the server room by different routes. The digital radio tuners received their input directly from RJ45/Cat8 Ethernet cables. The Motorolla DSR600 satellite tuners get their input from a satellite dish on the roof.
There are two types of radio tuner. The Grace Tuner (left) and the CC Radio Tuner (right). The CC Radio Tuner is newer, smaller, more reliable and more capable. These images show the rear of these devices and their respective I/O interfaces.
The Lenovo towers ingest analog audio and video using the expansion card models identified under "Existing office infrastructure" above. Each tower can capture from at most two analog TV feeds using separate expansion cards. This limit is based on previous reliability and longevity research performed by Aktiv Solutions. The ingestion of analog audio puts less stress on the hardware so multiple (currently at most six) audio feeds can be captured by a single tower. The images below illustrate the interface types and arrangements for these two cases. Media10 is shown on the left, media4 is on the right.
One of the towers (media8) captures both video and audio concurrently.
Digital video is captured using the Raspberry PI devices. As these share the form factor of a credit card there is no room internally for expansion hardware. For this reason separate HDMI capture cards are required for each channel (below left). An interstitial device (splitter) is also required between the HDMI source and the capture card (below right).
Prior to adopting the Raspberry PI as the digital video capture device, three additional Lenovo towers were purchased and configured for this purpose. These have more CPU and more memory than the other media boxes, but they were never able to reliably accomplish the task. They couldn't keep up with the video capture process for an entire 24 hour period. These machines are currently being used for purposes other than video ingestion, including Jorel2 testing.
The Raspberry PI devices proved to be much more capable at capturing digital video, likely because they store the video stream on their local solid-state drives. The three HD capture servers (unused) are shown below.
When a combination of hardware that adequately performs a task is identified, TNO will purchase multiple instances of those devices for future use. This mitigates the situation where newer models become incompatible with existing hardware over time and ensures that a new identical device can be swapped in if there is a failure. Two types of surplus hardware are shown below (Raspberry PIs and an HD video splitter).
The server room contains two banks of Mac Mini computers, some of which are over ten years old. These perform various tasks, including producing a time-based video editor usage report, receiving newspaper front page images, receiving newspaper content, running Azure scripts and executing a Jorel2 instance that processes the following production events:
| PL/SQL | RSS | LDAP | Alert | HTML | Pagewatcher | Syndication |
|---|
The two banks of OS/X boxes are shown below:
Video capture is hard on mechanical storage devices and creates a lot of heat in the motherboards of the capturing machines. Historically this has caused the failure of both hard drives and motherboards. This situation was mitigated by installing a powerful HVAC device in the server room, but the capturing machines are still vulnerable to failure. The risk of failure also increases as the hardware ages.
Within each video capture box there is a tight set of version/model dependencies linking the capture card, the motherboard and their associated drivers. A hard drive failure might be simple to address, but when a failure occurs in a motherboard or capture card it is very difficult to find contemporary models of either that work well together. This is the main reason why Scott is eager to replace these aging systems.
The media boxes that capture standard-definition analog video have a distinct advantage over the digital capture devices. When ffmpeg saves an analog video as an mpeg2 file, the closed-captioned text, for the stream, is embedded in the file. The text for each clip is extracted from the underlying stream, when the clip is created, using the ccextractor command line utility.
The extracted text is a reliable representation of the words spoken during the clip and is superior in many ways to the speech-to-text functionality offered by Azure. As news items for TNO reports and alerts are selected based on their textual content, it is important that the text stored in the database matches the spoken word content of a TV or Radio show as closely as possible. The closed captioning approach has two main advantages over Azure's speech-to-text solution:
- Accuracy
- Speed
Closed-captioned text is available when the clip is created, whereas text extraction using Azure requires the following steps:
- The clip is uploaded to the Azure server.
- Speech to text processing is performed on the clip in real-time, so a five-minute clip will take five minutes to process.
- The extracted text is then downloaded from Azure to the NAS.
- After receipt of the text, it must be checked for content and formatting before being added as a news item/transcript. While this step also applies to closed-captioned text, the number and scope of the edits is greater in this case.
The audio/video capture infrastructure operates using the same devices as those available in the consumer space. This approach requires a separate tuner/decoder for each channel being captured. Every device requires an input (coaxial or ethernet cable) and provides an output (analog audio or analog/digital video). This strategy results in the proliferation of cables and their associated front-end and back-end devices. For example, a single video channel requires:
- Input from the satellite dish on the roof of 617 Government St.
- A DSR600 satellite receiver (http://www.kusat.com/shaw-direct/DSR600-HD-System/)
- A coaxial cable leading from the DSR600's SD composite video output to the capture box
- A WinTV-HVR-1600 card in an expansion slot on the capture box (https://www.hauppauge.com/pages/products/data_hvr1600.html)
A separate DSR600 device is required for each TV channel because that's the only method, provided by Shaw, by which a single channel can be extracted from the satellite signal. An ideal solution would move the channel tuning requirement into a single device that also ran the video capture process. While this would expose a single point of failure, a duplex failover solution is likely achievable.
The proliferation of tuners/decoders introduces an increased administrative overhead, most notably each device must be tuned to the correct channel. Most modern A/V devices can only be fully programmed using an infrared remote control. Some manufacturers segment the frequency ranges of their remote controls so that there is no overlap between devices while others do not. In the worst case, a single remote control could affect all the devices from the same manufacturer. Without blocking the IR detectors on devices other than the one you want to configure there's a danger that more than one device will respond to IR commands. This situation could be avoided if a single device could capture multiple channels and maintain the tuning configuration in persistent storage.
If the server room power supply is interrupted, much of the TNO functionality will cease to run. The website will continue to operate, as it is located in a data centre in Kamloops, but many of its data ingestion streams will cease to function. When the power supply resumes, every device in the room must reboot and begin running the same task(s) it was undertaking prior to the interruption. The list below outlines some of the issues that might arise during this recovery process:
- Tuners/Decoders may revert to their default channel settings and require the manual selection of the correct channel.
- Servers may select a plug-and-play configuration that is different from the one they were using prior to the reboot. This may render their capture cards inaccessible to their respective operating systems.