Create a service to summarize a text using an existing model (generic template)¶

This tutorial shows how to implement a Service in the Swiss AI Center project step by step. The Service is a text summarizer tool that summarizes a text using the Hugging Face library.

Introduction¶

This tutorial shows how to implement a Service that uses an existing model to summarize a text from Hugging Face.

Prerequisites¶

To follow this tutorial, we highly recommend you to follow the Getting started guide first.

It contains all the required tools to follow this tutorial.

Bootstrap the service based on the generic template¶

In this section, you will bootstrap a new service based on the Create a new service (generic) template.

You have three ways to bootstrap a new service based on the template:

Explaining the template¶

In this section, you will learn about the different files and folders that are part of the template.

README.md¶

This file contains a checklist of the steps to follow to bootstrap a new service based on the template. This can help you to follow step-by-step what you need to do to bootstrap a new service based on the template.

Other files and folders¶

The other files and folders contain everything to serve your tool. This is where you will implement to do it over a FastAPI REST API.

Implement the service¶

Create a virtual environment¶

Instead of installing the dependencies globally, it is recommended to create a virtual environment.

To create a virtual environment, run the following command inside the project folder:

1
2

# Create a virtual environment
python3.11 -m venv .venv

Then, activate the virtual environment:

1
2

# Activate the virtual environment
source .venv/bin/activate

Install the dependencies¶

Update the requirements.txt file with the following content:

1
2
3

common-code[test] @ git+https://github.com/swiss-ai-center/common-code.git@main
transformers==4.39.3
torch==2.2.2

The common-code package is required to serve the model over a FastAPI REST API and boilerplate code to handle the configuration.

We will also need to install transformers and torch for the service to work.

Install the dependencies as explained in the previous section:

1
2

# Install the dependencies
pip install --requirement requirements.txt

Create a freeze file to pin all dependencies to their current versions:

1
2

# Freeze the dependencies
pip freeze --local --all > requirements-all.txt

The specific common-code @ git+https://github.com/swiss-ai-center/common-code.git@<commit> line will conflict with the more general line in requirements.txt due to the explicit commit reference.

From there, you have two options:

• Easier update: Remove the specific common-code @ git+https://github.com/swiss-ai-center/common-code.git@<commit> line from requirements-all.txt. This allows for easier updates of the common-code dependency without adjusting service dependencies.
• Consistent dependencies: Remove the generic line in requirements.txt to keep dependencies consistent across machines and time. This will ensure that the same versions of the dependencies are installed on every machine if you ever share your code with someone else.

Update the template files¶

Update the README¶

Open the README.md file and update the title and the description of the Service.

1
2
3

# Text Summarizer

This service summarizes a text using the Hugging Face library.

1
2
3
4
5

# Text Summarizer

This service summarizes a text using the Hugging Face library.

_Check the [related documentation](https://swiss-ai-center.github.io/swiss-ai-center/reference/text-summarizer) for more information._

Update the pyproject.toml file¶

1
2
3
4
5
6

[project]
name = "text-summarizer"

[tool.pytest.ini_options]
pythonpath = [".", "src"]
addopts = "--cov-config=.coveragerc --cov-report xml --cov-report term-missing --cov=./src"

1. Change the name of the project to text-summarizer.

Update the src/main.py file¶

All the code of the Service is in the main.py file. The Service is a text summarizer tool that summarizes a text using the Hugging Face library.

Open the main.py with your favorite editor and follow the instructions below.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121

import asyncio
import time
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import RedirectResponse
from common_code.config import get_settings
from common_code.http_client import HttpClient
from common_code.logger.logger import get_logger, Logger
from common_code.service.controller import router as service_router
from common_code.service.service import ServiceService
from common_code.storage.service import StorageService
from common_code.tasks.controller import router as tasks_router
from common_code.tasks.service import TasksService
from common_code.tasks.models import TaskData
from common_code.service.models import Service
from common_code.service.enums import ServiceStatus
from common_code.common.enums import FieldDescriptionType, ExecutionUnitTagName, ExecutionUnitTagAcronym
from common_code.common.models import FieldDescription, ExecutionUnitTag
from contextlib import asynccontextmanager

# Imports required by the service's model
# TODO: 1. ADD REQUIRED IMPORTS (ALSO IN THE REQUIREMENTS.TXT) (1)!
from transformers import pipeline

settings = get_settings()
classifier = pipeline("summarization", model="philschmid/bart-large-cnn-samsum")


class MyService(Service):
    # TODO: 2. CHANGE THIS DESCRIPTION (2)!
    """
    Text summarizer model
    """

    # Any additional fields must be excluded of model by adding a leading underscore for Pydantic to work
    _model: object
    _logger: Logger

    def __init__(self):
        super().__init__(
            # TODO: 3. CHANGE THE SERVICE NAME AND SLUG (3)!
            name="Text Summarizer",
            slug="text-summarizer",
            url=settings.service_url,
            summary=api_summary,
            description=api_description,
            status=ServiceStatus.AVAILABLE,
            # TODO: 4. CHANGE THE INPUT AND OUTPUT FIELDS, THE TAGS AND THE HAS_AI VARIABLE (4)!
            data_in_fields=[
                FieldDescription(name="text", type=[FieldDescriptionType.TEXT_PLAIN]),
            ],
            data_out_fields=[
                FieldDescription(name="result", type=[FieldDescriptionType.TEXT_PLAIN]),
            ],
            tags=[
                ExecutionUnitTag(
                    name=ExecutionUnitTagName.NATURAL_LANGUAGE_PROCESSING,
                    acronym=ExecutionUnitTagAcronym.NATURAL_LANGUAGE_PROCESSING,
                ),
            ],
            has_ai=True,
            docs_url="https://docs.swiss-ai-center.ch/reference/core-concepts/service/", # (5)!
        )
        self._logger = get_logger(settings)

    # TODO: 5. CHANGE THE PROCESS METHOD (CORE OF THE SERVICE) (6)!
    def process(self, data):
        # Get the text to analyze from storage
        text = data["text"].data
        # Convert bytes to string
        text = text.decode("utf-8")

        # Limit the text to 142 words
        text = " ".join(text.split()[:500])

        # Run the model
        result = classifier(text, max_length=100, min_length=5, do_sample=False)

        # Convert the result to bytes
        file_bytes = result[0]["summary_text"].encode("utf-8")

        return {
            "result": TaskData(
                data=file_bytes,
                type=FieldDescriptionType.TEXT_PLAIN
            )
        }

...

# TODO: 6. CHANGE THE API DESCRIPTION AND SUMMARY (7)!
api_summary = """
Summarize the given text.
"""

api_description = """
Summarize the given text using the HuggingFace transformers library with model bart-large-cnn-samsum.
"""

# Define the FastAPI application with information
# TODO: 7. CHANGE THE API TITLE, VERSION, CONTACT AND LICENSE (8)!
app = FastAPI(
    lifespan=lifespan,
    title="Text summarizer API.",
    description=api_description,
    version="0.0.1",
    contact={
        "name": "Swiss AI Center",
        "url": "https://swiss-ai-center.ch/",
        "email": "info@swiss-ai-center.ch",
    },
    swagger_ui_parameters={
        "tagsSorter": "alpha",
        "operationsSorter": "method",
    },
    license_info={
        "name": "GNU Affero General Public License v3.0 (GNU AGPLv3)",
        "url": "https://choosealicense.com/licenses/agpl-3.0/",
    },
)
...

1. Import the required packages.
2. Change the description of the service.
3. Change the name and the slug of the service. This is used to identify the service in the Core engine.
4. Change the input/output fields of the service. The name of the field is the key of the dictionary that will be used in the process function. The type of the field is the type of the data that will be sent to the service. They are defined in the FieldDescriptionType enum. The tags are used to identify the service in the Core engine. The has_ai variable is used to identify if the service is an AI service.
5. Optional: Edit the documentation URL of the service.
6. Change the process function. This is the core of the service. The data is a dictionary with the keys being the field names set in the data_in_fields. The result must be a dictionary with the keys being the field names set in the data_out_fields.
7. Change the API description and summary. The description is a markdown string that will be displayed in the API documentation. The summary is a short description of the API.
8. Change the API title, version, contact and license.

Start the service¶

1
2
3
4
5

# Build the Docker image
docker compose build

# Start the Core engine Backend
docker compose up

1
2
3
4
5

# Switch to the `src` directory
cd src

# Start the application
uvicorn --reload --host 0.0.0.0 --port 9090 main:app

The service should try to announce itself to the Core engine.

It will then be available at http://localhost:9090.

Access the Core engine either at http://localhost:3000 (Frontend UI) or http://localhost:9090 (Backend UI).

The service should be listed in the Services section.

Test the service¶

Create a file called text.txt and add some text in it.

There are two ways to test the service:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

{
  "created_at": "2024-01-09T16:30:22.627620",
  "updated_at": "2024-01-09T16:54:58.221528",
  "data_in": [
    "8a3bb409-bfe6-444d-9f94-8e245b0370b6.txt"//(1)!
  ],
  "data_out": null,//(2)!
  "status": "pending",
  "service_id": "c0ddda6c-9ffa-4912-90a2-b4e9b1769b56",
  "pipeline_execution_id": null,
  "id": "b7b73c95-88d1-4f02-87f6-29feabf41d7e",//(3)!
  "service": {
    "created_at": "2024-01-09T16:30:22.627620",
    "updated_at": "2024-01-09T16:49:08.499620",
    "description": "\nSummarize the given text using the HuggingFace transformers library with model bart-large-cnn-samsum.\n",
    "status": "available",
    "data_in_fields": [
      {
        "name": "text",
        "type": [
          "text/plain"
        ]
      }
    ],
    "data_out_fields": [
      {
        "name": "result",
        "type": [
          "text/plain"
        ]
      }
    ],
    "tags": [
      {
        "name": "Natural Language Processing",
        "acronym": "NLP"
      }
    ],
    "has_ai": true,
    "id": "c0ddda6c-9ffa-4912-90a2-b4e9b1769b56",
    "name": "Text Summarizer",
    "slug": "text-summarizer",
    "summary": "\nSummarize the given text.\n",
    "url": "http://localhost:9090",
    "docs_url": "https://docs.swiss-ai-center.ch/reference/services/text-summarizer/",
  },
  "pipeline_execution": null
}

You have validated that the service works as expected.

Commit and push the changes (optional)¶

Commit and push the changes to the Git repository so it is available for the other developers.

Build, publish and deploy the service¶

Now that you have implemented the service, you can build, publish and deploy it.

Follow the How to build, publish and deploy a service guide to build, publish and deploy the service to Kubernetes.

Access and test the service¶

Access the service using its URL (either by the URL defined in the DEV_SERVICE_URL/ PROD_SERVICE_URL variable or by the URL defined in the Kubernetes Ingress file).

You should be able to access the FastAPI Swagger UI.

The service should be available in the Services section of the Core engine it has announced itself to.

You should be able to send a request to the service and get a response.

Conclusion¶

Congratulations! You have successfully created a service to summarize a text.

The service has then been published to a container registry and deployed on Kubernetes.

The service is now accessible through a REST API on the Internet and you have completed this tutorial! Well done!