Deploy ​

Environment setup ​

To deploy the solution you can use 3 different methods:

1. AWS Cloud9 (Recommended)
2. Github Codespaces
3. Local machine

AWS Cloud9 ​

We recommend deploying with AWS Cloud9.

Use the Cloud9 console to create a new Cloud9 instance. Ensure you use the following values when creating the instance:

• Select m5.large or larger as Instance Type.
• Select Ubuntu Server 22.04 LTS as Platform.

The default EBS volume create with the Cloud9 instance is too small and you need to increase it to at least 100GB. To do this, run the following command from the Cloud9 terminal:

git clone https://github.com/aws-samples/aws-genai-llm-chatbot.git
cd aws-genai-llm-chatbot/
chmod +x scripts/cloud9-resize.sh
./scripts/cloud9-resize.sh

git clone https://github.com/aws-samples/aws-genai-llm-chatbot.git
cd aws-genai-llm-chatbot/
chmod +x scripts/cloud9-resize.sh
./scripts/cloud9-resize.sh

See the documentation for more details on environment resize.

You can now proceed with the deployment

Github Codespaces ​

To use GitHub Codespaces to deploy the solution, you need the following before proceeding:

1. An AWS account
2. An IAM User with AdministratorAccess policy granted (for production, we recommend restricting access as needed)

After creating the user, take note of Access Key ID and Secret Access Key.

Next, click on the button below to open your Codespaces environment.

Once in the Codespaces terminal, set up the AWS Credentials by running

aws configure

aws configure

AWS Access Key ID [None]: <the access key from the IAM user generated above>
AWS Secret Access Key [None]: <the secret access key from the IAM user generated above>
Default region name: <the region you plan to deploy the solution to>
Default output format: json

AWS Access Key ID [None]: <the access key from the IAM user generated above>
AWS Secret Access Key [None]: <the secret access key from the IAM user generated above>
Default region name: <the region you plan to deploy the solution to>
Default output format: json

You are all set for deployment; you can now jump to step 3 of the deployment section below.

Local machine ​

If are using a local machine, verify that your environment satisfies the following prerequisites:

You have:

1. An AWS account

2. An IAM User with AdministratorAccess policy granted (for production, we recommend restricting access as needed)

3. NodeJS 18 or 20 installed

   • If you are using nvm you can run the following before proceeding

     nvm install 18 && nvm use 18

     nvm install 18 && nvm use 18

     or

     nvm install 20 && nvm use 20

     nvm install 20 && nvm use 20

4. AWS CLI installed and configured to use with your AWS account

5. AWS CDK CLI installed

6. Docker installed

   • N.B. buildx is also required. For Windows and macOS buildx is included in Docker Desktop

7. Python 3+ installed

Deployment ​

Before you start, please read the precautions and security pages.

Step 1. Clone the repository.

git clone https://github.com/aws-samples/aws-genai-llm-chatbot

git clone https://github.com/aws-samples/aws-genai-llm-chatbot

Step 2. Move into the cloned repository.

cd aws-genai-llm-chatbot

cd aws-genai-llm-chatbot

Step 3. Install the project dependencies and build the project.

npm ci && npm run build

npm ci && npm run build

Step 4. (Optional) Run the unit tests

npm run test && pip install -r pytest_requirements.txt && pytest tests

npm run test && pip install -r pytest_requirements.txt && pytest tests

Step 5. Once done, run the configuration command to help you set up the solution with the features you need:

npm run config

npm run config

You'll be prompted to configure the different aspects of the solution, such as:

• The LLMs or MLMs to enable (we support all models provided by Bedrock that were enabled along with SageMaker hosted Idefics, FalconLite, Mistral and more to come).
• Setup of the RAG system: engine selection (i.e. Aurora w/ pgvector, OpenSearch, Kendra).
• Embeddings selection.
• Limit accessibility to website and backend to VPC (private chatbot).
• Add existing Amazon Kendra indices as RAG sources

When done, answer Y to create or update your configuration.

Your configuration is now stored under bin/config.json. You can re-run the npm run config command as needed to update your config.json

Step 6. (Optional) Bootstrap AWS CDK on the target account and region

Note: This is required if you have never used AWS CDK on this account and region combination. (More information on CDK bootstrapping).

npm run cdk bootstrap aws://{targetAccountId}/{targetRegion}

npm run cdk bootstrap aws://{targetAccountId}/{targetRegion}

You can now deploy by running:

npm run cdk deploy

npm run cdk deploy

Note: This step duration can vary greatly, depending on the Constructs you are deploying.

You can view the progress of your CDK deployment in the CloudFormation console in the selected region.

Step 7. Once deployed, take note of the User Interface, User Pool and, if you want to interact with 3P models providers, the Secret where to store API_KEYS to access 3P model providers.

...
Outputs:
GenAIChatBotStack.UserInterfaceUserInterfaceDomanNameXXXXXXXX = dxxxxxxxxxxxxx.cloudfront.net
GenAIChatBotStack.AuthenticationUserPoolLinkXXXXX = https://xxxxx.console.aws.amazon.com/cognito/v2/idp/user-pools/xxxxx_XXXXX/users?region=xxxxx
GenAIChatBotStack.ApiKeysSecretNameXXXX = ApiKeysSecretName-xxxxxx
...

...
Outputs:
GenAIChatBotStack.UserInterfaceUserInterfaceDomanNameXXXXXXXX = dxxxxxxxxxxxxx.cloudfront.net
GenAIChatBotStack.AuthenticationUserPoolLinkXXXXX = https://xxxxx.console.aws.amazon.com/cognito/v2/idp/user-pools/xxxxx_XXXXX/users?region=xxxxx
GenAIChatBotStack.ApiKeysSecretNameXXXX = ApiKeysSecretName-xxxxxx
...

Step 8. Open the generated Cognito User Pool Link from outputs above i.e. https://xxxxx.console.aws.amazon.com/cognito/v2/idp/user-pools/xxxxx_XXXXX/users?region=xxxxx

Step 9. Add a user that will be used to log into the web interface.

Step 10. Open the User Interface Url for the outputs above, i.e. dxxxxxxxxxxxxx.cloudfront.net.

Step 11. Login with the user created in Step 8 and follow the instructions.

Step 12. (Optional) Run the integration tests The tests require to be authenticated against your AWS Account because it will create cognito users. In addition, the tests will use anthropic.claude-instant-v1 (Claude Instant), anthropic.claude-3-haiku-20240307-v1:0 (Claude 3 Haiku) and amazon.titan-embed-text-v1 (Titan Embeddings G1 - Text) which need to be enabled in Bedrock.

To run the tests (Replace the url with the one you used in the steps above)

REACT_APP_URL=https://dxxxxxxxxxxxxx.cloudfront.net pytest integtests/ --ignore integtests/user_interface -n 3 --dist=loadfile

REACT_APP_URL=https://dxxxxxxxxxxxxx.cloudfront.net pytest integtests/ --ignore integtests/user_interface -n 3 --dist=loadfile

To run the UI tests, you will fist need to download and run geckodriver

REACT_APP_URL=https://dxxxxxxxxxxxxx.cloudfront.net pytest integtests/user_interface

REACT_APP_URL=https://dxxxxxxxxxxxxx.cloudfront.net pytest integtests/user_interface

Monitoring ​

Once the deployment is complete, a Amazon CloudWatch Dashboard will be available in the selected region to monitor the usage of the resources.

For more information, please refer to the monitoring page

Run user interface locally ​

To experiment with changes to the the user interface, you can run the interface locally. See the instructions in the README file of the lib/user-interface/react-app folder.

Using Kendra with a non-english index ​

If you're using Kendra with an index in a language other than English, you will need to make some code modifications.

You'll need to modify the filters in the file lib/shared/layers/python-sdk/python/genai_core/kendra/query.py.

Example for french :

    if kendra_index_external or kendra_use_all_data:
        result = kendra.retrieve(
            IndexId=kendra_index_id,
            QueryText=query,
            PageSize=limit,
            PageNumber=1,
            AttributeFilter={'AndAllFilters': [{"EqualsTo": {"Key": "_language_code","Value": {"StringValue": "fr"}}}]}
        )
    else:
        result = kendra.retrieve(
            IndexId=kendra_index_id,
            QueryText=query,
            PageSize=limit,
            PageNumber=1,
            AttributeFilter={'AndAllFilters':
                [
                    {"EqualsTo": {"Key": "_language_code","Value": {"StringValue": "fr"}}},
                    {"EqualsTo": {"Key": "workspace_id","Value": {"StringValue": workspace_id}}}
                ]
            }
        )

    if kendra_index_external or kendra_use_all_data:
        result = kendra.retrieve(
            IndexId=kendra_index_id,
            QueryText=query,
            PageSize=limit,
            PageNumber=1,
            AttributeFilter={'AndAllFilters': [{"EqualsTo": {"Key": "_language_code","Value": {"StringValue": "fr"}}}]}
        )
    else:
        result = kendra.retrieve(
            IndexId=kendra_index_id,
            QueryText=query,
            PageSize=limit,
            PageNumber=1,
            AttributeFilter={'AndAllFilters':
                [
                    {"EqualsTo": {"Key": "_language_code","Value": {"StringValue": "fr"}}},
                    {"EqualsTo": {"Key": "workspace_id","Value": {"StringValue": workspace_id}}}
                ]
            }
        )

Important: After you have done these changes it's essential to redeploy the solution:

npx cdk deploy

npx cdk deploy

Clean up ​

You can remove the stacks and all the associated resources created in your AWS account by running the following command:

npx cdk destroy

npx cdk destroy

Note: Depending on which resources have been deployed. Destroying the stack might take a while, up to 45m. If the deletion fails multiple times, please manually delete the remaining stack's ENIs; you can filter ENIs by VPC/Subnet/etc using the search bar here in the AWS console) and re-attempt a stack deletion.