how can i set the milvus log level

[huangxueliang]

the milvus docker log is too big, how can i set the log level?

[yhmo]

I think it is a docker issue. Search "How to clean Docker container logs?" to get the answer.
Milvus log level default is info, defined in the milvus.yaml:

# Configures the system log output.
log:
  # Milvus log level. Option: debug, info, warn, error, panic, and fatal. 
  # It is recommended to use debug level under test and development environments, and info level in production environment.
  level: info
  file:
    # Root path to the log files.
    # The default value is set empty, indicating to output log files to standard output (stdout) and standard error (stderr).
    # If this parameter is set to a valid local path, Milvus writes and stores log files in this path.
    # Set this parameter as the path that you have permission to write.
    rootPath: 
    maxSize: 300 # The maximum size of a log file, unit: MB.
    maxAge: 10 # The maximum retention time before a log file is automatically cleared, unit: day. The minimum value is 1.
    maxBackups: 20 # The maximum number of log files to back up, unit: day. The minimum value is 1.
  format: text # Milvus log format. Option: text and JSON
  stdout: true # Stdout enable or not

To change the configuration, follow this doc: https://milvus.io/docs/configure-docker.md?tab=component#Configure-Milvus-with-Docker-Compose

[huangxueliang]

i have done as you said.

[huangxueliang]

there are still have "info" log,how can i do?

[huangxueliang]

[yhmo]

Did you change the milvus.yaml?

If all is done, I have no idea. Maybe bring your machine to my office and I can setup everything for you.

[huangxueliang]

i have changed the milvus.yaml

[huangxueliang]

[yhmo]

v2.4.6 is also no problem.
Just node: the server prints a few info/debug logs at the initialization stage, which is expected behavior because the log level didn't take effect at the initialization state. After the server is ready, you call interfaces from client, all the logs are WARN.

[huangxueliang]

[huangxueliang]

but i think it is not initialization state info.

[yhmo]

I used v2.4.6 to test.

Firstly, I start the milvus with log.level=info and docker compose up -d command, do some insert/search:
Use command docker logs [container id] > 1.log to output the logs:
1.log

In the 1.log, we can see there are INFO "checkpoint from latest consumed msg" inside the logs:
[2024/11/12 03:59:10.638 +00:00] [INFO] [writebuffer/write_buffer.go:263] ["checkpoint from latest consumed msg"] [collectionID=453871912753037344] [channel=by-dev-rootcoord-dml_0_453871912753037344v0]

Then I stop the milvus and remove its container by command docker compose down

And set log.level=warn, use docker compose up -d command to start a new milvus container, do some insert/search:
Use command docker logs [container id] > 2.log to output the logs:
2.log

In the 2.log, we can see there are no INFO/DEBUG after this line:
[2024/11/12 04:00:45.562 +00:00] [INFO] [components/proxy.go:58] ["Proxy successfully started"]

Because the level=warn takes effect after initialization. All the log following this line are [WARN].

My docker-compose.yml and milvus.yaml for testing:
docker-compose.txt
milvus.txt

My script for testing:

import random
from pymilvus import (
     connections,
     utility,
     FieldSchema, CollectionSchema, DataType,
     Collection,
 )

dim = 128
collection_name = "AAA"


if __name__ == "__main__":
    connections.connect()

    id_field = FieldSchema(name="id", dtype=DataType.INT64, is_primary=True, auto_id=False)
    vector_field = FieldSchema(name="vector", dtype=DataType.FLOAT_VECTOR, dim=dim)
    schema = CollectionSchema(fields=[id_field, vector_field])

    has = utility.has_collection(collection_name)
    if has:
        utility.drop_collection(collection_name)

    collection = Collection(collection_name, schema)

    nb = 10000
    data = [
        [i for i in range(nb)],
        [[random.random() for _ in range(dim)] for i in range(nb)],
    ]
    collection.insert(data)
    collection.flush()
    print("inserted")

    index_params = {
        "index_type": "FLAT",
        "metric_type": "COSINE",
        "params": {},
    }
    collection.create_index("vector", index_params=index_params)
    print("indexed")

    collection.load()
    print("loaded")

    collection = Collection(collection_name)
    collection.load()

    vectors = [[random.random() for _ in range(dim)], [random.random() for _ in range(dim)]]
    topk = 10
    results_future = collection.search(data=vectors,
                                       limit=topk,
                                       param={
                                           "metric_type": "COSINE",
                                       },
                                       output_fields=["id"],
                                       anns_field="vector",
                                       expr="id > 5000",
                                       _async=True)

    results = results_future.result()
    for res in results:
        print(f"\nTop{topk} results:")
        for r in res:
            print(r)

[huangxueliang]

thank you very much!