Skip to content

Commit

Permalink
make service.mount_to_api lazy by default, fix db monitor issue on my…
Browse files Browse the repository at this point in the history
…sql, add requires check for db backends, update log query scope, fix cache adaptor
  • Loading branch information
voidZXL committed Nov 21, 2024
1 parent 9b08d80 commit 9be484c
Show file tree
Hide file tree
Showing 33 changed files with 320 additions and 111 deletions.
33 changes: 31 additions & 2 deletions docs/zh/guide/ops.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,11 @@ Operations 配置项的主要参数包括
如果你不希望让 OperationsAPI 与你的其他 API 使用同一个域,你的 `route` 参数也可以指定为一个绝对路径,例如 `route='https://ops.mysite.com'` 表示通过 `https://ops.mysite.com` 来提供 OperationsAPI ,这种情况你需要自行处理 OperationsAPI 的挂载与前端代理

* `database`:必填,设置管理系统存储日志,监控等运维数据的数据库,你可以向上面的例子一样指定一个 SQLite 数据库,在生产环境中也可以指定一个 `postgresql` 数据库

!!! tip
如果你需要配置 MySQL 或 PostgreSQL 数据库,请记得指定数据库的地址,用户名,密码等信息


* `base_url`:为你的 API 服务指定一个可以在网络上访问到的基准 API 地址,这个地址会用于生成的 OpenAPI 文档的 `server.url`,设置后 OperationsAPI 的地址 = `base_url` + `route`

!!! note
Expand Down Expand Up @@ -289,7 +294,7 @@ meta init --app=django_settings.wsgi.app
app = django_settings.wsgi.app
```

UtilMeta 框架就是靠这个文件识别 UtilMeta 项目,以及项目的核心对象的地址的
UtilMeta 框架就是靠这个文件识别 UtilMeta 项目,以及项目的核心应用对象的地址的

### 连接 Django

Expand All @@ -313,12 +318,18 @@ Operations(
engine='sqlite3'
# or 'postgres' / 'mysql' / 'oracle'
),
base_url='https://blog.mysite.com/api',
base_url='https://<YOUR DOMAIN>/api',
# base_url='http://127.0.0.1:<YOUR_PORT>', # 本地项目
).integrate(application, __name__)
```

我们先声明了 `Operations` 配置,参数如上文介绍的一样,然后调用了配置实例的 `integrate` 方法,第一个参数传入 WSGI / ASGI 应用,第二个参数传入 `__name__`

需要注意的是 `Operations` 配置的 `base_url` 参数需要提供你的 Django 服务的 **基准访问地址**,也就是你的 API 服务中定义的路径都会从这个地址延申,如果是部署在网络上的服务,请设置为能够在网络中访问到的 URL,如果是本地服务,则设置为 `http://127.0.0.1:你的端口号

!!! note
如果你需要配置 MySQL 或 PostgreSQL 数据库,请记得指定数据库的地址,用户名,密码等信息`

!!! tip
如果你使用了 **Django REST framework**,UtilMeta 将会自动同步 DRF 生成的 OpenAPI 文档

Expand Down Expand Up @@ -347,12 +358,18 @@ Operations(
engine='sqlite3' # or 'postgresql' / 'mysql'
),
base_url='https://<YOUR DOMAIN>/api',
# base_url='http://127.0.0.1:<YOUR_PORT>', # 本地项目
).integrate(app, __name__)
```

!!! tip
如果你使用了 **APIFlask**,也只需要把 Operations 配置接入 APIFlask 的 app 中,UtilMeta 将会自动同步 APIFlask 生成的 OpenAPI 文档

`Operations` 配置的 `base_url` 参数需要提供你的 Flask 服务的 **基准访问地址**,也就是你的 API 服务中定义的路径都会从这个地址延申,如果是部署在网络上的服务,请设置为能够在网络中访问到的 URL,如果是本地服务,则设置为 `http://127.0.0.1:你的端口号`

!!! note
如果你需要配置 MySQL 或 PostgreSQL 数据库,请记得指定数据库的地址,用户名,密码等信息

加入配置代码后,如果你的项目是本地运行,可以在重启项目后执行如下命令连接本地服务

```
Expand All @@ -377,12 +394,18 @@ Operations(
engine='sqlite3' # or 'postgresql' / 'mysql'
),
base_url='https://<YOUR DOMAIN>/api',
# base_url='http://127.0.0.1:<YOUR_PORT>', # 本地项目
).integrate(app, __name__)
```

!!! tip
UtilMeta 将自动同步 FastAPI 生成的 API 文档

`Operations` 配置的 `base_url` 参数需要提供你的 FastAPI 服务的 **基准访问地址**,也就是你的 API 服务中定义的路径都会从这个地址延申,如果是部署在网络上的服务,请设置为能够在网络中访问到的 URL,如果是本地服务,则设置为 `http://127.0.0.1:你的端口号`

!!! note
如果你需要配置 MySQL 或 PostgreSQL 数据库,请记得指定数据库的地址,用户名,密码等信息

加入配置代码后,如果你的项目是本地运行,可以在重启项目后执行如下命令连接本地服务

```
Expand All @@ -408,12 +431,18 @@ Operations(
engine='sqlite3' # or 'postgresql' / 'mysql'
),
base_url='https://<YOUR DOMAIN>/api',
# base_url='http://127.0.0.1:<YOUR_PORT>', # 本地项目
).integrate(app, __name__)
```

!!! tip
UtilMeta 将自动同步 Sanic 的 openapi 扩展生成的 API 文档

`Operations` 配置的 `base_url` 参数需要提供你的 Sanic 服务的 **基准访问地址**,也就是你的 API 服务中定义的路径都会从这个地址延申,如果是部署在网络上的服务,请设置为能够在网络中访问到的 URL,如果是本地服务,则设置为 `http://127.0.0.1:你的端口号`

!!! note
如果你需要配置 MySQL 或 PostgreSQL 数据库,请记得指定数据库的地址,用户名,密码等信息

加入配置代码后,如果你的项目是本地运行,可以在重启项目后执行如下命令连接本地服务

```
Expand Down
8 changes: 6 additions & 2 deletions docs/zh/tutorials/user-auth.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ meta setup demo-user
```
在提示选择 backend 的时候输入 `django`

项目创建好后,我们需要先对服务的数据库连接进行配置,打开 `server.py`插入以下代码
项目创建好后,我们需要先对服务的数据库连接进行配置,打开 `server.py``service` 的定义和 `app` 的定义之间插入以下代码

```python
service = UtilMeta(...)
Expand All @@ -35,6 +35,8 @@ service.use(DatabaseConnections({
engine='sqlite3',
)
}))
# ----------------
app = service.application()
```

在插入的代码中,我们声明了 Django 的配置信息与数据库连接的配置
Expand Down Expand Up @@ -440,7 +442,9 @@ if __name__ == '__main__':

其中编写了用户注册接口和获取当前用户接口的调试代码,当我们启动服务并运行 `test.py` 时,我们可以看到的输出类似

```json
```
> python test.py
Response [200 OK] "POST /api/user/signup"
application/json (76)
{'username': 'user1', 'id': 1, 'signup_time': '2024-01-29T12:29:33.684594'}
Expand Down
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ license = {file = "LICENSE"}
authors = [
{ name = "Xulin Zhou (voidZXL)", email = "zxl@utilmeta.com" },
]
keywords = ["API", "backend", "orm", "RESTful", "meta", "progressive", "declarative", "web", "utype"]
keywords = ["API", "backend", "orm", "RESTful", "meta", "progressive", "declarative", "web", "utype", "devops"]
classifiers = [
"Intended Audience :: Information Technology",
"Intended Audience :: System Administrators",
Expand All @@ -24,7 +24,7 @@ classifiers = [
"Topic :: Software Development :: Libraries",
"Topic :: Software Development",
"Typing :: Typed",
"Development Status :: 3 - Alpha",
"Development Status :: 4 - Beta",
"Environment :: Web Environment",
"Intended Audience :: Developers",
"License :: OSI Approved :: Apache Software License",
Expand Down
4 changes: 3 additions & 1 deletion tests/server/app/schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,9 @@ class ContentSchema(orm.Schema[BaseContent]):
auth={'r': auth.Require()}
) # test read auth, test common field mount
created_at: datetime
updated_at: datetime = Field(default_factory=datetime.now, no_input='aw') # last modified
# updated_at: datetime = Field(default_factory=datetime.now, no_input='aw') # last modified
updated_at: datetime
# test auto assignment
public: bool = orm.Field(auth={
# 'w': orm.Relate('author'),
'a': auth.Require()
Expand Down
File renamed without changes.
29 changes: 29 additions & 0 deletions tests/test_0_basic/test_cache.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
import pytest
from tests.conftest import setup_service

setup_service(__name__, backend='django', async_param=[False])


class TestCache:
def test_cache(self, service):
from utilmeta.core.cache import Cache
cache = Cache(
engine='memory'
)
assert cache.get('key') is None
cache.set('key', '123')
assert cache.get('key') == '123'
cache.pop('key')
assert cache.get('key') is None

# @pytest.mark.asyncio
# async def test_async_cache(self):
# from utilmeta.core.cache import Cache
# cache = Cache(
# engine='memory'
# )
# assert await cache.aget('key') is None
# await cache.aset('key', '123')
# assert await cache.aget('key') == '123'
# await cache.apop('key')
# assert await cache.aget('key') is None
File renamed without changes.
12 changes: 11 additions & 1 deletion tests/test_1_orm/test_schema_query.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import pytest
from tests.conftest import setup_service
from utilmeta.core import orm
from utilmeta.utils import exceptions
from utilmeta.utils import exceptions, time_now
from datetime import datetime
from typing import List, Optional

Expand Down Expand Up @@ -280,10 +280,19 @@ def test_save(self):
)
assert article.creatable_field == 'a'
assert article.slug == 'my-new-article-1'
t = time_now()
article.save()
t1 = time_now()
inst = article.get_instance(fresh=True)
assert t1 > inst.created_at > t
assert t1 > inst.updated_at > t
article.content = 'my new content'
article.save() # test save on mode 'a' with pk (should update instead of create)

# inst = article.get_instance(fresh=True)
# t2 = time_now()
# assert t2 > inst.updated_at > t1

with pytest.raises(exceptions.BadRequest):
article.save(must_create=True)

Expand Down Expand Up @@ -317,6 +326,7 @@ def test_save(self):
with pytest.raises(orm.MissingPrimaryKey):
article2.save(must_update=True)

assert article2.updated_at > t1
# article.pk = None
# article.save(must_create=True)
# assert inst.pk != article.pk
Expand Down
2 changes: 1 addition & 1 deletion tests/test_7_ops/django_asgi_site/django_demo/asgi.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,5 +55,5 @@ async def sleep(self, a: float) -> PlusResponse[float]:
name='operations_db',
),
base_url=f'http://127.0.0.1:{PORT}',
eager=True
eager_migrate=True
).integrate(service)
2 changes: 1 addition & 1 deletion tests/test_7_ops/django_asgi_site/django_demo/wsgi.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,5 @@
engine='sqlite3'
),
base_url='http://127.0.0.1:9091',
eager=True # eager migration for test
eager_migrate=True # eager migration for test
).integrate(application, __name__)
2 changes: 1 addition & 1 deletion tests/test_7_ops/django_site/django_demo/wsgi.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,5 @@
engine='sqlite3'
),
base_url='http://127.0.0.1:9091',
eager=True # eager migration for test
eager_migrate=True # eager migration for test
).integrate(application, __name__)
2 changes: 1 addition & 1 deletion tests/test_7_ops/fastapi_site/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ async def read_item(item_id) -> dict:
secure_only=False,
trusted_hosts=['127.0.0.1'],
base_url=f'http://127.0.0.1:{PORT}',
eager=True
eager_migrate=True
).integrate(app, __name__)


Expand Down
2 changes: 1 addition & 1 deletion tests/test_7_ops/flask_site/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ def get_pet(pet_id):
engine='sqlite3'
),
base_url=f'http://127.0.0.1:{PORT}',
eager=True
eager_migrate=True
).integrate(app, __name__)

if __name__ == '__main__':
Expand Down
2 changes: 1 addition & 1 deletion tests/test_7_ops/sanic_site/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ def get(self):
engine='sqlite3'
),
base_url=f'http://127.0.0.1:{PORT}',
eager=True
eager_migrate=True
).integrate(app, __name__)

if __name__ == '__main__':
Expand Down
2 changes: 1 addition & 1 deletion tests/test_7_ops/tornado_site/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ def make_app():
engine='sqlite3'
),
base_url=f'http://127.0.0.1:{PORT}',
eager=True
eager_migrate=True
).integrate(application, __name__)
return application

Expand Down
2 changes: 1 addition & 1 deletion tests/test_7_ops/utilmeta_site/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ def hello(self):
name='operations_db',
engine='sqlite3' # or 'postgres' / 'mysql' / 'oracle'
),
eager=True
eager_migrate=True
# base_url='http://127.0.0.1:{}/'.format(PORT),
))

Expand Down
2 changes: 1 addition & 1 deletion utilmeta/__init__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
__website__ = 'https://utilmeta.com'
__homepage__ = 'https://utilmeta.com/py'
__author__ = 'Xulin Zhou (@voidZXL)'
__version__ = '2.6.2'
__version__ = '2.6.3'


def version_info() -> str:
Expand Down
4 changes: 2 additions & 2 deletions utilmeta/conf/env.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,8 @@ def _load_from_file(self) -> Mapping:
return json.load(open(self._file, 'r'))

if self._file.endswith('.yml') or self._file.endswith('.yaml'):
from utilmeta.utils import check_requirement
check_requirement('pyyaml', install_when_require=True)
from utilmeta.utils import requires
requires(yaml='pyyaml')
import yaml
return yaml.safe_load(open(self._file, 'r'))

Expand Down
Loading

0 comments on commit 9be484c

Please sign in to comment.