Skip to content

A server code for serving BERT-based models for text classification. It is designed by SerpApi for heavy-load prototyping and production tasks, specifically for the implementation of the google-local-results-ai-parser gem.

License

Notifications You must be signed in to change notification settings

serpapi/google-local-results-ai-server

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

32 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Google Local Results AI Server

Contributors Forks Stargazers Issues Issues MIT License

AI Model Server

This repository contains the code for a server that mimics the Inference API endpoints at Huggingface for BERT-based classification models. The server provides a simple interface to perform text classification using BERT models. It is designed by SerpApi for the heavy-load prototyping, and production tasks for the implementation of google-local-results-ai-parser gem which uses serpapi/bert-base-local-results model.

Relevant Sources

Installation

To set up and run the Google Local Results AI Server locally, follow these steps:

  • Clone the repository:
cd google-local-results-ai-server
git clone git clone https://huggingface.co/serpapi/bert-base-local-results ./google/bert-base-local-results
  • Create and activate a virtual environment (optional but recommended):
python3 -m venv env
source env/bin/activate
  • Install the dependencies:
pip install -r requirements.txt
  • Set the necessary environment variables at main.py:
# Master key to keep track of access
MASTER_KEY = "master_key"
# Make sure you have git-lfs installed (https://git-lfs.com)
git lfs install
git lfs pull
git clone https://huggingface.co/serpapi/bert-base-local-results google/bert-base-local-results
  • Start the server using gunicorn:
gunicorn main:app --workers 4 --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0

You may set the number of workers depending on your system allowace. The server will start running on http://localhost:8000.


Docker Deployment

Alternatively, you can deploy the Google Local Results AI Server using Docker. The repository already contains a Dockerfile for easy deployment.

A Docker Image will be created and published in the repository.

To build the image, and deploy the server with Docker locally, follow these steps:

  • Set the necessary environment variables at main.py:
# Master key to keep track of access
MASTER_KEY = "master_key"
  • Build the Docker image:
docker build -t google-local-results-ai-server .
  • **Optional: Change the number of workers depending on your system allowance in Dockerfile. Here's an example with 2 workers:

    • CMD ["gunicorn", "main:app", "--workers", "2", "--worker-class", "uvicorn.workers.UvicornWorker", "--bind", "0.0.0.0"]
  • Run the Docker container:

docker run -p 8000:8000 google-local-results-ai-server

The server will start running on http://localhost:8000.


Usage

The server exposes an HTTP API endpoint that accepts POST requests for classification. You can send requests to the API endpoint using various programming languages and tools. The following examples demonstrate how to make requests using Ruby, Python, JavaScript, and cURL.

Ruby Example

require 'http'

url = "http://127.0.0.1:8000/models/serpapi/bert-base-local-results"
headers = { "Authorization" => "Bearer XXXXXX" }
payload = { "inputs" => "5540 N Lamar Blvd #12 Austin, TX 78756" }

response = HTTP.headers(headers).post(url, json: payload)
output = response.parse

Python Example

import requests

API_URL = "http://127.0.0.1:8000/models/serpapi/bert-base-local-results"
headers = {"Authorization": "Bearer XXXXXX"}

def query(payload):
    response = requests.post(API_URL, headers=headers, json=payload)
    return response.json()

output = query({
    "inputs": "5540 N Lamar Blvd #12 Austin, TX 78756",
})

JavaScript Example

async function query(data) {
    const response = await fetch(
        "http://127.0.0.1:8000/models/serpapi/bert-base-local-results",
        {
            headers: { Authorization: "Bearer XXXXXX" },
            method: "POST",
            body: JSON.stringify(data),
        }
    );
    const result = await response.json();
    return result;
}

query({"inputs": "5540 N Lamar Blvd #12 Austin, TX 78756"}).then((response) => {
    console.log(JSON.stringify(response));
});

cURL Example

curl http://127.0.0.1:8000/models/serpapi/bert-base-local-results \
    -X POST \
    -d '{"inputs": "5540 N Lamar Blvd #12 Austin, TX 78756"}' \
    -H "Authorization: Bearer XXXXXX"

Example JSON Output

[
  [
    {
      "label": "address",
      "score": 0.9988067150115967
    },
    {
      "label": "type",
      "score": 0.0010613144841045141
    },
    {
      "label": "description",
      "score": 0.00021563934569712728
    },
    {
      "label": "hours",
      "score": 0.0002154999820049852
    },
    {
      "label": "phone",
      "score": 0.00018228559929411858
    },
    {
      "label": "reviews",
      "score": 0.00007934834866318852
    },
    {
      "label": "service options",
      "score": 0.0000681085730320774
    },
    {
      "label": "price",
      "score": 0.00001069890731741907
    },
    {
      "label": "years in business",
      "score": 0.000007037287559796823
    },
    {
      "label": "button text",
      "score": 0.000006214133918547304
    },
    {
      "label": "rating",
      "score": 0.000004599460226017982
    }
  ]
]

Advanced Usage

The server code provided in this repository supports advanced usage, allowing you to extend the models served by adding more BERT-based classification models. However, please note that the existing models are hardcoded with their paths in the code. The current model, google/bert-base-local-results, is located at ./google/bert-base-local-results.

To add more models, follow these steps:

  • Prepare the BERT-based classification model and save it in a directory.
  • Update the models dictionary in the code to include the new model name and its corresponding directory path.
models = {
    "serpapi/bert-base-local-results": "./google/bert-base-local-results",
    "new-repository/new-model": "new-model-folder-path"
}
  • Restart the server.

The server will automatically load the new models at startup, and you can access them using the appropriate API endpoints.

Please note that you need to ensure the model directory contains the necessary files for the BERT-based classification model, such as the model weights, configuration, and tokenizer.

About

A server code for serving BERT-based models for text classification. It is designed by SerpApi for heavy-load prototyping and production tasks, specifically for the implementation of the google-local-results-ai-parser gem.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published