Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update real allocated size of a volume, after it is created #1

Merged
merged 1 commit into from
Nov 29, 2023

Conversation

silentred
Copy link
Collaborator

@silentred silentred commented Nov 29, 2023

When following the quick start document, the LVS type volume cannot be succussfully created.

To make sture that a storagepool exists

kubectl -n obnvmf get storagepool
NAME                                               IP              HOSTNAME                                           STORAGE   FREE    STATUS   AGE
ip-172-26-14-129.ap-northeast-1.compute.internal   172.26.14.129   ip-172-26-14-129.ap-northeast-1.compute.internal   998Mi     998Mi   ready    54m

Show the detailed info of the storagepool

kubectl -n obnvmf get storagepool ip-172-26-14-129.ap-northeast-1.compute.internal -oyaml
apiVersion: volume.antstor.alipay.com/v1
kind: StoragePool
metadata:
  annotations:
    obnvmf/hostnqn: nqn.2021-03.com.alipay.host:uuid:ip-172-26-14-129.ap-northeast-1.compute.internal
    obnvmf/tgt-version: ""
  creationTimestamp: "2023-11-29T08:06:30Z"
  finalizers:
  - antstor.alipay.com/in-state
  generation: 1
  labels:
    obnvmf/local-storage-bytes: "1046478848"
    obnvmf/node-sn: ip-172-26-14-129.ap-northeast-1.compute.internal
  name: ip-172-26-14-129.ap-northeast-1.compute.internal
  namespace: obnvmf
  resourceVersion: "5713"
  uid: 20d1c020-c53a-45b2-8753-aa2b1b06bd75
spec:
  addresses:
  - address: 172.26.14.129
    type: InternalIP
  kernelLvm: {}
  nodeInfo:
    hostname: ip-172-26-14-129.ap-northeast-1.compute.internal
    id: ip-172-26-14-129.ap-northeast-1.compute.internal
    ip: 172.26.14.129
    labels:
      beta.kubernetes.io/arch: amd64
      beta.kubernetes.io/os: linux
      custom.k8s.alipay.com/obnvmf-host: "true"
      kubernetes.io/arch: amd64
      kubernetes.io/hostname: ip-172-26-14-129.ap-northeast-1.compute.internal
      kubernetes.io/os: linux
      node-role.kubernetes.io/control-plane: ""
      node.kubernetes.io/exclude-from-external-load-balancers: ""
  spdkLVStore:
    baseBdev: test-aio-bdev
    blockSize: 512
    bytes: 1046478848
    clusterSize: 2097152
    name: aio-lvs
    totalDataClusters: 499
    uuid: fa6ac6bd-89cd-4b03-a365-44ae7ff3c7e8
status:
  capacity:
    storage: 998Mi
  conditions:
  - status: OK
    type: Spdk
  status: ready
  vgFreeSize: 998Mi

the disk-agent daemonset pod has logs

E1129 08:13:18.190071       1 sync_loop.go:128] obnvmf/pvc-9c1af30c-5156-4597-9ae8-67e68f2e2137Code=-19 Msg=No such device
I1129 08:13:32.689461       1 heartbeat.go:37] Heartbeat is ok
I1129 08:13:48.192599       1 sync_loop.go:102] process key obnvmf/pvc-9c1af30c-5156-4597-9ae8-67e68f2e2137
I1129 08:13:48.196292       1 sync_volume.go:96] syncing volume pvc-9c1af30c-5156-4597-9ae8-67e68f2e2137 to create logic vol and spdk target
I1129 08:13:48.196313       1 sync_volume.go:167] volume pvc-9c1af30c-5156-4597-9ae8-67e68f2e2137 is not in expansion
2023/11/29 08:13:48 client.go:54: {"jsonrpc":"2.0","method":"bdev_get_bdevs","id":26,"params":{"name":"aio-lvs/pvc-9c1af30c-5156-4597-9ae8-67e68f2e2137"}}
2023/11/29 08:13:48 client.go:65:
E1129 08:13:48.202368       1 service.go:119] Code=-19 Msg=No such device
E1129 08:13:48.202384       1 lvs.go:94] Code=-19 Msg=No such device
E1129 08:13:48.202392       1 sync_volume.go:106] Code=-19 Msg=No such device
E1129 08:13:48.202402       1 sync_loop.go:120] key obnvmf/pvc-9c1af30c-5156-4597-9ae8-67e68f2e2137 has retry too many times 6, delay it for 30s

The problem is at this line:

vol, err = vs.poolService.PoolEngine().GetVolume(lvName)

If the PoolEngine is SpdkLVS and the Lvol is not created, GetVolume returns an error. This causes the volume hangs in creating status.

@silentred silentred merged commit 7184e03 into eosphoros-ai:main Nov 29, 2023
1 check passed
silentred added a commit that referenced this pull request Jan 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant