Skip to content

Home

Jump to bottom
JPEvain edited this page Oct 21, 2019 · 49 revisions

MCMA DEMO LIVING DOCUMENTATION

MCMA DEMO DEPENDENCIES

1. Node.js v8.10.0 installed and accessible in PATH. Recommended is to use a node version manager, which allows you to quickly switch between node versions (see more info at nvm-windows for windows support or nvm for Mac OS and Linux support). Note: lambdas can be associated with different versions of NodeJs to meet the requirements of specific libraries.

2. Install nvm-windows from https://github.com/coreybutler/nvm-windows/releases -> nvmsetup.zip/nvm-setup.exe

  • Cmd -> nvm –version
  • Cmd -> where nvm

If needed add to System environment variables:

  • NVN_HOME ...\AppData\Roaming\nvm
  • NVM SYM_LINK ...\Program Files\nodejs
  • RESTART IF NEEDED

3. Check AWS runtime versions for compatibility: https://github.com/coreybutler/nvm-windows/releases

  • Cmd -> nvm install 8.10.0 (current version supported by AWS)
  • Cmd -> node –version Node.js version 8.10.0
  • Cmd -> npm –version Npm version v5.6.0
  • if no answer Cmd -> where npm (should be under nodejs). If modules empty: there can be installation issues.
  • Cmd -> nvm use 8.10.0 to use this version (REQUIRED)

4. Github e.g. from gitforwindows.org

5. Latest version of Terraform and available in PATH. See the Terraform website (Create directory c:\terraform move zip’s terraform.exe in this directory and Update PATH environment variable with c:\terraform)

6. Java JRE or JDK 1.8 or higher to run Gradle build and deploy scripts

7. Create AWS account

8. Azure video indexer account, a free account can be used for testing. Follow these instructions: https://docs.microsoft.com/en-us/azure/cognitive-services/video-indexer/video-indexer-use-apis

FULL DEMO EXPERIENCE FROM THE AZURE_STTBENCHMARKING BRANCH

1. Clone repo from that branch and gradlew deploy from that directory in your local repo.

2. Full demo: AWS transcribe + translate English to French) + subtitles (SRT + VTT for the HTML5 media player) + celebrity identification + emotions + text-to-Speech using Polly and AZURE all metadata, incl. transcribe and celebrity identification. STT benchmarking is implemented on both branches.

3. Video: The full experience works only for video 2015_GF_ORF_00_18_09_conv.mp4 to be copied from the mcma media repository into the upload repository on AWS S3.

4. Translation requires a correct transcription output file. The reference transcription is provided for 2015_GF_ORF_00_18_09_conv.mp4. There is no editor in the demo and the file is preloaded and used when appropriate as the workflow executes. For instance, STT_OUTPUT from AWS transcribe is replaced by STT_OUTPUT_CLEAN for use by AWS translate (for obvious performance reasons). For the same reason, a clean subtitle file is provided for the translated file for use for text-to-speech timing. The two files stt_output_clean.txt and srt_translation_output.srt are preloaded with data contained in the website folder (respectively in the asset stt and srt folders) when the project is deployed using Gradle.

5. Setup procedure

  • Clone this repository to your local harddrive: Cmd -> “git clone https://github.com/ebu/mcma-projects or from the repo online and /or using the github app.
  • Navigate to the multi-cloud-ai-workflow folder.
  • Select the azure_sttbenchmarking branch
  • Create file named gradle.properties
  • Add the following information to the created file and update the parameter values reflecting your AWS account and Azure account. `

environmentName=ch.ebu.mcma.1

environmentType=ibc

awsAccountId=xxxxxxxxxxx

awsAccessKey=yyyyyyyyyyy

awsSecretKey=zzzzzzzzzz

awsRegion=eu-west-1

awsInstanceType=t2.micro

awsInstanceCount=1

AzureLocation=trial

AzureAccountID=aaaaaaaaaaa

AzureSubscriptionKey=bbbbbbbbbbbb

AzureApiUrl=https://api.videoindexer.aiSave the file.

`

  • Open command line in multi-cloud-ai-workflow folder.
  • Execute gradlew deploy and let it run. This can take a few minutes. (Execute gradlew destroy to uninstall e.g. if demo is not used)
  • If no error has occurred until now you have successfully setup the infrastructure in your AWS cloud. Go to https://aws.amazon.com/console/ and sign with AWSAccountID above (IAM user name =”admin” + pwd=”F1m$”) in to see your cloud infrastructure. In case you do have errors it may be that your environmentName is either too long or not unique enough to guarantee unique names for your cloud resources e.g. bucket names.
  • The URL to the demo app is provided in the terminal during the deploy process -> see website_url -> copy and paste in browser
  • Without EC2: go to content preloaded in S3 bucket “mcma-media-files” and copy and paste from there to the “upload” repository.
  • website_url = https://s3-eu-west-1.amazonaws.com/ch.ebu.mcma.1.eu-west-1.ibc.website/index.html after deployment

6. AWS Console key services

  • S3 to monitor and manage the storage buckets
  • ECS to monitor tasks of docker services (e.g. sttbenchmarking)
  • Step functions to visualise workflows
  • Cloudwatch to visualise logs
  • Workflow step logs (step logs identified by the step number and service logs: e.g. ibc-aws-ai-service-worker

7. Updating workflow with new service: describe job and initiate call, run service, register job results and deploy

I - Workflow step "job description and call"

  • Goal: define a new workflow step, which will initiate a call to a service (e.g. AWS STT or a service deployed under AWS ECS like STTBenchmarking , etc.).
  • See how steps are numbered in the existing code:`

Workflow step X - job description and call

Service associated to the job

Workflow step X+1 - register job results

`

  • Go to your local github repository, create a branch for your own development and go to multi-cloud-ai-workflow/ai (if working on new ai services)
  • Copy and paste any of the existing steps (full directory). For instance, 02-extract-speech-to-text represents a basic structure. Renumber (MUST start with a unique 2 digit number)
  • Open src/index.js and customize the code

The index.js code is split in two sections:

  • Declaration of variables, functions, AWS utilities and repository buckets (e.g. the TempBucket but also possible the WebsiteBucket, etc.). The JOB_PROFILE_NAME is the link to the service (explained later). JOB_RESULT_PREFIX defines the repository where results (e.g. data files) will be returned under the output location bucket defined in the call. Some functions may require java script libraries to be declared in the package.json file with references to aws, local or npm packages (https://www.npmjs.com/)

  • The export.handler (lambda function handler), which will handle the job profile and from where the job will be created/started.

The job definition is to be adapted to your needs and preferences, in particular input and output locators. Look at other job definitions in other even steps to see how locators (bucket and keyprefix) can also be used for input files. The keyprefix defines a directory. The filename will be define in the service (see below).

The code found in 02-extract-speech-to-text/src/index is similar in all job description and call steps.

  • Update multi-cloud-ai-workflow/settings.gradle with new steps
  • Update multi-cloud-ai-workflow/workflows/build.gradle with new steps

II - Service definition

• If new group of services (e.g. like ame-service, aws-ai-service, azure-ai-service), declare new xxx-service.tf file under deployment/services/ xxx-service.tf -> e.g. duplicate Azure service declaration for Google and IBM • Services are defined in mutli-cloud-ai-workflow/services • Services have been arranged in different clusters like the aws-ai-service group, which we recommend to use when adding new aws ai services • When do you want to create and use a service? basically anytime you want to access and use an ai tool such as below

  1. Simple services without triggers a) createDubbingSrt: adding dubbed audio and subtitle srt track to source video b) detect-celebrities: detecting celebrities using the aws.rekognition service -> celebrity recognition c) detect-emotions: detecting emotions using the aws.rekognition service -> face detection d) translate-text: Translation service from STT output (clean) to translation text e) validate-speech-to-text: use EBU STT benchmarking toolset to evaluate the result of STT vs. a corrected clean output
  2. Services with additional process and sns triggers (monitoring asynchronous processes) a) Process-reko-results: special case of processing aws.rekognition results in a separate process
  3. Services with additional process and s3 triggers (monitoring asynchronous processes) / call service and get output from service / trigger process of the results. a) Ssml-text-to-speech: generating mp3 file from a SSML marked text file fed to AWS Polly. b) Text-to-Speech: generating mp3 file using polly and text from translation c) Tokenized-Text-To-Speech: extract speechmarks per sentence and generate SSML file for Polly d) Transcribe-audio: speech to text • To create a new service: HINT: take a look at services that may be similar to what you want to do and replicate/customize code
  4. Go to mutli-cloud-ai-workflow/services/(aws-ai-service)/worker/src/profiles
  5. Select and exiting service as a template (looking at the list of services above can give an idea to start with one or another, which can be modified any time, anyway, copy and paste, rename…
  6. Update the declarations and code to adapt the service to your needs after the necessary initialisation of the jobInput common to all services and giving access to data from the job creation
  7. It is vital to give a xxxservice.profileName and declare the module functions into the module.exports.
  8. Do not forget to update dependency declarations in each concerned service’s package.json
  9. Next step consists of updating the index.js under worker/src: define the dependencies, addProfiles and handleOperations as appropriate
  10. Update the s3-trigger (beware of the patterns and the note on how to EXTRACT UUID FROM FILENAME!!) and sns-trigger files

###III. REGISTER JOB OUPUT – manage results

  1. Each register step follows the same pattern a) Get the data / resources files coming as output from the service b) Consolidate resources as files at chosen locations and associations of the essences as locators in bmContent and provide httpEndpoints c) Copy data to bmContent properties when applicable d) Etc.
  2. Update multi-cloud-ai-workflow/settings.gradle with new steps
  3. Update multi-cloud-ai-workflow/workflows/build.gradle with new steps

###IV. DEPLOYMENT USING TERRAFORM

  1. Update multi-cloud-ai-workflow/deployment/registry/src/index a) Add new JOB_PROFILES b) Update the definition of the createServices function
  2. Update multi-cloud-ai-workflow/deployment/services (see above)
  3. Update a) multi-cloud-ai-workflow/deployment/workflows/ai.tf and define new step functions b) multi-cloud-ai-workflow/deployment/workflows/ai.json and update the “lambdas for AI workflow” definition by mimicking existing structures – do not forget the “AI workflow” vars at the end of the file

###V. TEST

  1. Gradlew deploy
  2. Copy and paste website url in browser
  3. Select video (if not available, go to AWS console and upload/copy video to the upload bucket)
  4. Run workflow (name and description are required) – if workflow is validated, click on “goto workflow”.
  5. In case of doubt, monitor progress from AWS console -> step function, could watch, etc.

###VI. ADDITIONAL INFORMATION

  1. Deploy service on ECS from docker image a) Example of service: sttBenchmarking from https://benchmarkstt.readthedocs.io/en/latest/INSTALL.html -> docker -> b) Better OS -> MAC OS c) Install docker – docs.docker.com/docker-for-mac/install/ d) docker run --name benchmarkstt -p 1234:8080 --rm benchmarkstt:latest e) install python 3.6+ https://wsvincent.com/install-python3-mac/ f) install AWS CLI : https://docs.aws.amazon.com/cli/latest/userguide/install-macos.html g) push docker image to ecr/ecs using aws cli https://docs.aws.amazon.com/AmazonECR/latest/userguide/ECR_AWSCLI.html#AWSCLI_get-login or follow instructions in following fargate youtube video… h) create service and task accessing docker image -> aws fargate -> https://www.youtube.com/watch?v=fmFlAWtKnGA • Use the repository URI when asked for the repository ID • If correctly installed, there should be a cluster associated to a service and a running task • Click on the task ID and note the public URL generated by ECS when the service has been created • Use the URL to make an JSON-RPC call to the service – for stt benchmarking, the port is 8080 and the directory is /api • Look at code provided on github in the validate-speech-to-text service and how the JSON-RPC function has been promisified to give access to the data returned from the sttbenchmarking service. Deploy troubles Error messages like xxx exists, etc. Go to AWS console and delete all manually: S3, step functions, lambda and in IAM (roles and policies)

Google Extension

  • Create new google-ai-service.tf file under deployment/services
  • Create new google steps under workflows/ai/61-extract-audio-google -> initialize with const JOB_PROFILE_NAME = "ExtractAudio"; const JOB_RESULTS_PREFIX = "ExtractAudioJobResults/"; andinputFile: event.data.mediaFileLocator
  • Create new multi-cloud-ai-workflow/services/google-ai-workflow/worer/src/profiles/extractAudio.js and add ffmpeg… plus update index.js (align profileName in index.js with profileName in init step, here 61.
  • Create new step workflows/ai/62-register-extract-audio-google to create output mp3 file associated to bmcontent as bmessence.
  • Update settings-gradle et build.gradle et build.gradle dans services Check presence ffmpeg
  • Prepare terraform deployment scripts (see above) o Update deployment/registry… job_profiles and create case “google_ai_service_url o Create deployment/services/google-ai-service.tf o Update ai.tf and ai.json with new google branch o Update output url dans deployment/main.tf
  • Update google credentials in Services/google-ai-service.tf, etc.
  • Define google account variables in deployment/variables.tf and in deployment/main.tf and in deployment/build.gradle and in root/gradle.properties Gradle.properties GoogleLocation= GoogleAccountID= GoogleSubscriptionKey= GoogleApiUrl=
Clone this wiki locally