Deploying and Using AlignScore
The authors of the AlignScore paper have made their code available on Github. We wrap their code (1) in a FastAPI application and expose APIs that can be called by AAQ. The FastAPI application can be deployed using a Docker container.
- Actually, we wrap a fork of their code
Configure AAQ to use AlignScore
You can use AlignScore instead of the default LLM-based content validation by setting
ALIGN_SCORE_METHOD
to AlignScore
in the .env
file (1) in the deployment/
folder.
- If you don't a .env file, make sure you have deployed the app as per instruction in Quick Setup
ALIGN_SCORE_METHOD=AlignScore
Other configuration
You can also change the threshold score below which the LLM response is considered not consistent with the provided context.
ALIGN_SCORE_THRESHOLD=0.7
See Testing on how to decide on this threshold.
Deploying the service
Follow the deployment instructions in Quick Setup.
On Step 6: Run docker-compose, run the following command instead:
docker compose -f docker-compose.yml -f docker-compose.dev.yml \
--profile alignScore -p aaq-stack up -d --build
Dev setup
If you are using the Dev setup, you can start the container manually using:
cd optional_components/alignScore
docker build -t alignscore-base . # (1)
docker run -p 5001:5001 --name align-score-local --detach 'alignscore-base'
- This step can take long time and even longer if you have a slow internet connection.
So if you have already generated the image previously, you can skip this step. Alternatively,
you can pull from ECR using
docker pull \ public.ecr.aws/j3r7b4k0/alignscore-base:latest