aws-scalable-big-blue-button-example

Status: Work-in-progress. Please create issues or pull requests if you have ideas for improvement.

Scalable Big Blue Button Video Conference Example

Example deployment of a scalable Video Conference System

Summary

This project demonstrates the deployment of a scalable Video Conference Setup jointly using open-source software components and AWS Services.

High-Level architecture

EC2 based deployment - architecture

Serverless based deployment - architecture

Disclaimer

This project is an example of an deployment and meant to be used for testing and learning purposes only. Do not use in production.

Please note that running this code will cause software that is licensed under AGPL-3.0 to be deployed in the user’s account.

Be aware that the deployment is not covered by the AWS free tier. Please use the AWS pricing calculator to an estimation beforehand

Getting started
Prerequisites
Parameters
Templates
Code updates
Versioning
Troubleshooting common errors
Resources
Security
License

Getting started

Just a few steps are needed to get started with the example deployment. the deployment process is separated in a prerequisites deployment containing the creation of the source file Amazon S3 Bucket and another containing the actual deployment of the infrastructure and application layer.

You may use the included setup script to simplify and automatic deployment or alternatively you can run the deployment step-by-step.

Prerequisites

To run the automated, setup script based deployment you need to have some software installed and configured on your device:

bash (zsh, csh, sh should also work, not tested though)
an installed and configured aws-cli
a named profile at the aws-cli configuration reflecting the account you are planning to use for the deployment
jq
docker

To run the step-by-step setup:

an installed and configured aws-cli

either way you choose the following have to be in place:

a valid Route53 Hosted Zone (and it’s ZoneID). Either a registered Domain or an external registered Domain (at the target account)
one or more valid Email Addresses
Amazon Simple Email Service needs to either being out of the Sandbox mode OR you need to validate destination email adresses piece by piece for the Invitation Emails

Be sure to check the Troubleshooting common errors when deploying.

Parameters

Dynamic parameters

These parameters you have to pass to the setup script

Parameter Name	Value
-p	the aws-cli profile to use
-e	the operators email address
-h	the hosted zone ID the DNS records to be added
-s	the Cloudformation stack name you want to use
-d	the FQDN for (aligned to the hosted zone )
-l	the log level choose DEBUG, INFO, WARN, ERROR

Deployment parameters:

The deployment parameters are placed into the bbb-on-aws-param.json or to be set via cli/console ( if you choose the step-by-step setup. )

Deployment

Automatic

For the automatic deployment just run the included setup script

Example:

./setup.sh -e johndoe@example.com -p bbb_example -h 1XXX02XXXXBMXXXXXZXXXX -s bbbexample -d bbbexample.example.com

The automatic deployment works as follows:

The setup script will validate the device prerequisites are met and all needed parameters are set.
It will then validate the syntax of the Amazon Cloudformation templates prior to execute any deployment.
It’s going to deploy the Amazon S3 Bucket needed by the main deployment and read out the Bucket name as well as the name of the Stack deployed.
It will copy the needed scripts, config files for application and services as well as nested templates to the the deployed Bucket.
The main deployment will be executed. The script will read the content of the bbb-on-aws-param.json file and pass it through the stack deployment

Step-by-step

If you want to attempt the deployment step-by-step via Console or aws-cli please use the following steps:

deploy the Source Amazon S3 Bucket for scripts, config files and nested templates

aws cloudformation deploy --stack-name bbbexample-sources --profile=bbb_example --template ./templates/bbb-on-aws-buildbuckets.template.yaml

copy the content of the templates abd scripts into the Source Bucket

    aws s3 sync --profile=bbb_example ./templates s3://NAMEOFCREATEDBUCKET
    aws s3 sync --profile=bbb_example ./scripts s3://NAMEOFCREATEDBUCKET

start the deployment using the stackname of the stack deployed beforehand as one of the parameters:

using aws-cli:

aws cloudformation deploy --profile=bbb_example --stack-name bbbexample \
    --capabilities CAPABILITY_NAMED_IAM \
    --parameter-overrides BBBOperatorEMail=johndoe@example.com BBBStackBucketStack=bbbexample-sources BBBDomainName=bbbexample.example.com BBBHostedZone=1XXX02XXXXBMXXXXXZXXXX \
    --template ./bbb-on-aws-master.template.yaml

The deployment will take approx 30-45 minutes.

Logging into the Big Blue Button:

When deployment went through you find the Administrator login within Secretsmanager. The password is located at the secret starting with BBBAdministratorlogin- While your login has been set to your selected Operator Email Address the password has been generated for you.

log into the Frontend using conference.example.com

Template structure and deployment workflow

The Deployment consists of 2 main templates and 13 nested templates.

Main templates

The deployment of prerequisites via bbb-on-aws-buildbuckets.template.yaml

The template deploys the Amazon S3 Bucket containing the scripts used at the turn, application and scalelite deployment as well as the nested templates source files.
The Master Template for the main deployment bbb-on-aws-master.template.yaml

The template initiates the overall deployment of the scaling Big Blue Button example deployment.

Nested templates
Providing the Certificate Automation via: bbb-on-aws-acmcert.template.yaml

This template deploys a custom resource into your AWS Account which provides full automation of requesting and validating Amazon Certificate manager based TLS Certificates. The validation will be done via Route53 DNS records.
Dynamically provide the latest AMI to be used with: bbb-on-aws-amifinder.template.yaml

The template deploys a custom resource into your AWS Account which will access the AWS API to find the ami of the desired non-Amazon Linux Operating System.
Setup Email sending via Amazon SES: bbb-on-aws-ses.template.yaml

This template deploys a custom resource which fully automates the Amazon SES configuration for sending mails and validating the domains in Amazon SES via Route53 records.
Building the network infrastructure using: bbb-on-aws-network.template.yaml

The template deploys a full Amazon VPC with public and private subnets, route tables, internet gateways, nat gateways.
Create needed Security Groups: bbb-on-aws-securitygroups.template.yaml

This template creates all needed security groups for service exposure as well as application backend communication
Provisioning of the Shared Storage: bbb-on-aws-storage.template.yaml

This template deploys an Amazon Elastic File System into the setup to provide shared storage for the video conference recordings
Deploy Amazon Aurora (Postgres): bbb-on-aws-database.template.yaml

The deployment of Amazon Aurora is needed to provide a database for Greenlight and Scalelite where the video conference schedules, user data and recording information are persistent
Deploy Amazon Elasticache: bbb-on-aws-cachedb.template.yaml

This template deploys an Amazon Elasticache cluster where security token and conference IDs are located for the call handling via Scalelite
Fire up the ECS Cluster: bbb-on-aws-ecs.template.yaml

The template deploys the ECS cluster and EC2 Autoscaling Group with the Launch Configuration for the Amazon EC2 worker nodes. If the parameter BBBECSInstanceType is set to “fargate” the ECS Cluster will be utilizing AWS Fargate for the tasks and EC2 worker instances as well as Autoscaling Groups will not be created
Initiate the frontend and inner-application load balancer to the ECS Cluster: bbb-on-aws-frontendapps.template.yaml

Greenlight is providing the interface for the Users and the video conference landing page. Scalelite is the inner-application load balancer which provides conference load balancing over multiple big blue button application servers
Setting up the Big Blue Button Application Instance(s): bbb-on-aws-bbbappscalable.template.yaml (for single server deployments bbb-on-aws-bbbappsingle.template.yaml)

Finally we will deploy the Big Blue Button application instances. Each instance is basically a separate video conference system.

Custom scripts

During the deployment the EC2 instances will be bootstrapped using UserData.

route53-handler systemd service and script deployed to the application and turn server instances creates dynamic records in your hosted zone (Route53) on boot/bootstrap of an instance and removes the record from the hosted zone.
We do not use ElasticIPs or fixed hostnames at the setup to prevent e.g. service quotas. Also we use dynamic hostnames for each bootstrapped instance to prevent e.g. Let’s encrypt penalties —
scalelite-handler systemd service and script *deployed to each application instance adds the instance to scalelite on boot/bootstrap as active video conference instance and evaluates/removes the instance on shutdown” —
scalelite post script and config the script and config files are added to the Big Blue Button application instances to enable the recordings import into scalelite and Amazon EFS

Configuration adjustments (diffs from the defaults)

Cloudwatch Agent for application instances the agent is automatically setup via UserData on bootstrap. To sent valid data to the Amazon Cloudwatch Service a custom agent config is used for application and turn servers

Customizing your Big Blue Button deployment

There are several ways how you can further customize your deployment. Apart from the infrastructure components you can customize using the parameters mentioned earlier at the documentation you can also adjust the bootstrap of the Big Blue Button or Greenlight deployment according to your needs. A good starting point is to take a look at the UserData Section of the nested stack for the application instances like: bbb-on-aws-bbbappscalable.template.yaml (for single server deployments bbb-on-aws-bbbappsingle.template.yaml)

Our recommendation is to hook into the bootstrap and alter/extend the Scripts and/or code there. this makes sure your customization will be persistent for all of your deployments and also if you decide to scale-out the application servers. Basically the customization section of the Big Blue Button documentation does content all steps you need.

When it comes to Greenlight there is also a part at the official documentation covering this. As we do use the containerized version of the Greenlight deployment at the scalable option the best way to approach it is to customize and extend the related Greenlight container, push it into your private container registy. Amazon ECR or any docker compatible registry of your choise. And continue with your customized container image setting the related parameter.

Integrating with Learning Management Systems

Integrating the Big Blue Button Setup into a LMS is possible. As example to integrate with Moodle a few steps need to be taken

Use the parameter “BBBFrontendType” and set it to “External”
Follow the instructions to setup the BBB Plugin into Moodle
the Big Blue Button Server URL should reflect your Scalelite ALB endpoint like https://scalelite.example.com
to get the needed Big Blue Button Shared Secret visit the AWS Secrets Manager and look for the secret called “BBBLoadbalancerSecret”. Look into the “basekeyvalue” key. that’s whaat you need to add to the addon setup.

Important, with the scalable setup used with Scalelite recordings of sessions are transferred to the central Shared Storage at Amazon EFS. Your LMS setup should either mount the volume or you need to change that strategy. Please read through the Amazon EFS documentation to get the needed access and prerequisites setup we’re using AccessPoints and IAM support to grant least privileges and secure access. You might want to adjust the template to your needs (Roles added for access as an example).

Code updates

to update an already deployed stack just pull the current version of the IaC code repository. Afterwards you can start the upgrade process the same way as you would do the initial setup.

Versioning

We’re using the Semantic Versioning for this repo. Each major release will be tagged and can be pulled seperately. Be sure NOT to use the main branch if you want to be sure not pulling potential huge changes to the infrastrucutre unintentionally. Use the branches regarding to the major Version you want to stick to.

Troubleshooting common errors

Failed to create: [BBBApplicationAutoScaling] Issue:

One of the most common errors is, that the Hosted Zone metioned at the Prerequisites is not setup properly or you’re at the state of DNS delay. Use the following commands on your command line to evaluate if your DNS Setup is working:

nslookup thedomain.setupashosted.zone

This should reply with some basic domain information like assigned DNS Servers. If you get a domain not found error wait a bit if you’re sure you followed the docomentation above ( registered Domain or an external registered Domain (at the target account) ) and try like 10min later. If still no results, check your hosted zone and DNS setup. If you use an external Registar (and did not buy a domain using Route53) make sure you registered the Route53 DNS Servers with the Domain as mentioned at the related documentation.

You do need an own domain to deploy in any case. you cannot mock it using bbb.example.com or any invalid setup here.
Failed to create: [BBBECSCapacityProvider] Issue:

There are some cases where the needed Service Role is non-existent at the target account. Solution: Create the missing Service Role manually using the following aws-cli command:

aws iam create-service-linked-role --aws-service-name ecs.amazonaws.com

Resources

AWS Services
Open Source Projects

Security

See CONTRIBUTING for more information.

License

This Example is licensed under the MIT-0 License. See the LICENSE file.

3rd party Licensing

Please be aware of the deviating licenses of the deployed open-source software components.

Big Blue Button: GNU Lesser General Public License v3.0
Greenlight: GNU Lesser General Public License v3.0
Scalelite: GNU Affero General Public License v3.0
cli53: MIT License
Coturn: New BSD License
cfn-ses-provider - Apache-2.0 License

Please note that running this code will cause software that is licensed under AGPL-3.0 to be deployed in the user’s account.

This site is open source. Improve this page.

aws-scalable-big-blue-button-example

Scalable Big Blue Button Video Conference Example

Summary

High-Level architecture

EC2 based deployment - architecture

Serverless based deployment - architecture

Disclaimer

Table of Contents

Getting started

Prerequisites

Parameters

Deployment

Automatic

Step-by-step

Logging into the Big Blue Button:

Template structure and deployment workflow

Main templates

The template deploys the Amazon S3 Bucket containing the scripts used at the turn, application and scalelite deployment as well as the nested templates source files.

The template initiates the overall deployment of the scaling Big Blue Button example deployment.

Nested templates

This template deploys a custom resource into your AWS Account which provides full automation of requesting and validating Amazon Certificate manager based TLS Certificates. The validation will be done via Route53 DNS records.

The template deploys a custom resource into your AWS Account which will access the AWS API to find the ami of the desired non-Amazon Linux Operating System.

This template deploys a custom resource which fully automates the Amazon SES configuration for sending mails and validating the domains in Amazon SES via Route53 records.

The template deploys a full Amazon VPC with public and private subnets, route tables, internet gateways, nat gateways.

This template creates all needed security groups for service exposure as well as application backend communication

This template deploys an Amazon Elastic File System into the setup to provide shared storage for the video conference recordings

The deployment of Amazon Aurora is needed to provide a database for Greenlight and Scalelite where the video conference schedules, user data and recording information are persistent

This template deploys an Amazon Elasticache cluster where security token and conference IDs are located for the call handling via Scalelite

Greenlight is providing the interface for the Users and the video conference landing page. Scalelite is the inner-application load balancer which provides conference load balancing over multiple big blue button application servers

Finally we will deploy the Big Blue Button application instances. Each instance is basically a separate video conference system.