Is client-side sorting really disabled?

[ddovod]

Hi. I'm experiencing weird behaviour with company-lsp-cache-candidates variable.
I'm using cquery + emacs-cquery + lsp-mode + company-lsp.
The thing is cquery sorts completion candidates itself and I don't want company to sort it, so I'm setting company-lsp-cache-candidates to nil to disable caching and sorting (as documentation describes), but looks like completion candidates are sorted without respect to this parameter (company-lsp--on-completion calls lsp--sort-completions without check of company-lsp-cache-candidates value). You can find more details in this pull request in the pre-last message jacobdufault/cquery#786 .
So could you please help me with it? Thank you very much!

[ddovod]

After some investigation I found that company-lsp uses some cache for candidates and sorts them no matter what value is in company-lsp-cache-candidates. Am I correct? Can I disable client-side sorting somehow?

[tigersoldier]

Per specification, the server can specify sortText for each candidate. Candidates should be sorted by sortText if the server populates them, or sorted by the labels if sortText is not populated.

This is more like a cquery bug if sortText is provided. Can you open a bug on cquery side instead?

To check if sortText is provided, try M-x company-diag at the point to be completed, and find if "sortText" appears in the output.

[ddovod]

Hey, thank you for your response!
cquery sets sortText this way (after all items being sorted(:

    char buf[16];
    for (size_t i = 0; i < items.size(); ++i) {
        items[i].sortText = tofixedbase64(i, buf);
    }

so this field contains ordered alphanumeric string like (first 5 completion candidates)

Completions:
  #("strcat" 0 6 (lsp-completion-prefix #("str" 0 1 (fontified t face (highlight-symbol-face)) 1 2 (fontified t face (highlight-symbol-face)) 2 3 (fontified t face (highlight-symbol-face))) lsp-completion-item #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("label" "strcat" "kind" 3 "detail" "char * strcat(char *__restrict __dest, const char *__restrict __src)" "documentation" "" "sortText" "..........." "insertText" "strcat(${1:char *__restrict __dest}, ${2:const char *__restrict __src})$0" "filterText" "strcat" "insertTextFormat" 2 "textEdit" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("range" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("start" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 13)) "end" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 16)))) "newText" "strcat(${1:char *__restrict __dest}, ${2:const char *__restrict __src})$0")))))) " char * strcat(char *__restrict __dest, const char *__restrict __src) (Function)"
  #("strchr" 0 6 (lsp-completion-prefix #("str" 0 1 (fontified t face (highlight-symbol-face)) 1 2 (fontified t face (highlight-symbol-face)) 2 3 (fontified t face (highlight-symbol-face))) lsp-completion-item #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("label" "strchr" "kind" 3 "detail" "char * strchr(char *__s, int __c)" "documentation" "" "sortText" "........../" "insertText" "strchr(${1:char *__s}, ${2:int __c})$0" "filterText" "strchr" "insertTextFormat" 2 "textEdit" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("range" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("start" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 13)) "end" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 16)))) "newText" "strchr(${1:char *__s}, ${2:int __c})$0")))))) " char * strchr(char *__s, int __c) (Function)"
  #("strchr" 0 6 (lsp-completion-prefix #("str" 0 1 (fontified t face (highlight-symbol-face)) 1 2 (fontified t face (highlight-symbol-face)) 2 3 (fontified t face (highlight-symbol-face))) lsp-completion-item #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("label" "strchr" "kind" 3 "detail" "const char * strchr(const char *__s, int __c)" "documentation" "" "sortText" "..........0" "insertText" "strchr(${1:const char *__s}, ${2:int __c})$0" "filterText" "strchr" "insertTextFormat" 2 "textEdit" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("range" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("start" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 13)) "end" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 16)))) "newText" "strchr(${1:const char *__s}, ${2:int __c})$0")))))) " const char * strchr(const char *__s, int __c) (Function)"
  #("strchr" 0 6 (lsp-completion-prefix #("str" 0 1 (fontified t face (highlight-symbol-face)) 1 2 (fontified t face (highlight-symbol-face)) 2 3 (fontified t face (highlight-symbol-face))) lsp-completion-item #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("label" "strchr" "kind" 3 "detail" "char * strchr(const char *__s, int __c)" "documentation" "" "sortText" "..........1" "insertText" "strchr(${1:const char *__s}, ${2:int __c})$0" "filterText" "strchr" "insertTextFormat" 2 "textEdit" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("range" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("start" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 13)) "end" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 16)))) "newText" "strchr(${1:const char *__s}, ${2:int __c})$0")))))) " char * strchr(const char *__s, int __c) (Function)"
  #("strcmp" 0 6 (lsp-completion-prefix #("str" 0 1 (fontified t face (highlight-symbol-face)) 1 2 (fontified t face (highlight-symbol-face)) 2 3 (fontified t face (highlight-symbol-face))) lsp-completion-item #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("label" "strcmp" "kind" 3 "detail" "int strcmp(const char *__s1, const char *__s2)" "documentation" "" "sortText" "..........2" "insertText" "strcmp(${1:const char *__s1}, ${2:const char *__s2})$0" "filterText" "strcmp" "insertTextFormat" 2 "textEdit" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("range" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("start" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 13)) "end" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 16)))) "newText" "strcmp(${1:const char *__s1}, ${2:const char *__s2})$0")))))) " int strcmp(const char *__s1, const char *__s2) (Function)"
  #("strcpy" 0 6 (lsp-completion-prefix #("str" 0 1 (fontified t face (highlight-symbol-face)) 1 2 (fontified t face (highlight-symbol-face)) 2 3 (fontified t face (highlight-symbol-face))) lsp-completion-item #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("label" "strcpy" "kind" 3 "detail" "char * strcpy(char *__restrict __dest, const char *__restrict __src)" "documentation" "" "sortText" "..........3" "insertText" "strcpy(${1:char *__restrict __dest}, ${2:const char *__restrict __src})$0" "filterText" "strcpy" "insertTextFormat" 2 "textEdit" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("range" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("start" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 13)) "end" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 16)))) "newText" "strcpy(${1:char *__restrict __dest}, ${2:const char *__restrict __src})$0")))))) " char * strcpy(char *__restrict __dest, const char *__restrict __src) (Function)"
  #("string" 0 6 (lsp-completion-prefix #("str" 0 1 (fontified t face (highlight-symbol-face)) 1 2 (fontified t face (highlight-symbol-face)) 2 3 (fontified t face (highlight-symbol-face))) lsp-completion-item #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("label" "string" "kind" 7 "detail" "string" "documentation" "" "sortText" "..........4" "insertText" "string" "filterText" "string" "insertTextFormat" 1 "textEdit" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("range" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("start" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 13)) "end" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 16)))) "newText" "string")))))) " string (Class)"

But anyway company-lsp shows these items:

[ddovod]

Even if I set sortText as numeric string, the result is same

#("strcat" 0 6 (lsp-completion-prefix #("str" 0 1 (fontified t) 1 2 (fontified t) 2 3 (fontified t)) lsp-completion-item #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("label" "strcat" "kind" 3 "detail" "char * strcat(char *__restrict __dest, const char *__restrict __src)" "documentation" "" "sortText" "00" "insertText" "strcat(${1:char *__restrict __dest}, ${2:const char *__restrict __src})$0" "filterText" "strcat" "insertTextFormat" 2 "textEdit" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("range" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("start" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 13)) "end" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 16)))) "newText" "strcat(${1:char *__restrict __dest}, ${2:const char *__restrict __src})$0")))))) " char * strcat(char *__restrict __dest, const char *__restrict __src) (Function)"
  #("strchr" 0 6 (lsp-completion-prefix #("str" 0 1 (fontified t) 1 2 (fontified t) 2 3 (fontified t)) lsp-completion-item #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("label" "strchr" "kind" 3 "detail" "char * strchr(const char *__s, int __c)" "documentation" "" "sortText" "01" "insertText" "strchr(${1:const char *__s}, ${2:int __c})$0" "filterText" "strchr" "insertTextFormat" 2 "textEdit" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("range" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("start" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 13)) "end" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 16)))) "newText" "strchr(${1:const char *__s}, ${2:int __c})$0")))))) " char * strchr(const char *__s, int __c) (Function)"
  #("strchr" 0 6 (lsp-completion-prefix #("str" 0 1 (fontified t) 1 2 (fontified t) 2 3 (fontified t)) lsp-completion-item #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("label" "strchr" "kind" 3 "detail" "const char * strchr(const char *__s, int __c)" "documentation" "" "sortText" "02" "insertText" "strchr(${1:const char *__s}, ${2:int __c})$0" "filterText" "strchr" "insertTextFormat" 2 "textEdit" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("range" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("start" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 13)) "end" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 16)))) "newText" "strchr(${1:const char *__s}, ${2:int __c})$0")))))) " const char * strchr(const char *__s, int __c) (Function)"
  #("strchr" 0 6 (lsp-completion-prefix #("str" 0 1 (fontified t) 1 2 (fontified t) 2 3 (fontified t)) lsp-completion-item #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("label" "strchr" "kind" 3 "detail" "char * strchr(char *__s, int __c)" "documentation" "" "sortText" "03" "insertText" "strchr(${1:char *__s}, ${2:int __c})$0" "filterText" "strchr" "insertTextFormat" 2 "textEdit" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("range" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("start" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 13)) "end" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 16)))) "newText" "strchr(${1:char *__s}, ${2:int __c})$0")))))) " char * strchr(char *__s, int __c) (Function)"
  #("strcmp" 0 6 (lsp-completion-prefix #("str" 0 1 (fontified t) 1 2 (fontified t) 2 3 (fontified t)) lsp-completion-item #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("label" "strcmp" "kind" 3 "detail" "int strcmp(const char *__s1, const char *__s2)" "documentation" "" "sortText" "04" "insertText" "strcmp(${1:const char *__s1}, ${2:const char *__s2})$0" "filterText" "strcmp" "insertTextFormat" 2 "textEdit" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("range" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("start" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 13)) "end" #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ("line" 72 "character" 16)))) "newText" "strcmp(${1:const char *__s1}, ${2:const char *__s2})$0")))))) " int strcmp(const char *__s1, const char *__s2) (Function)"

[tigersoldier]

Are the items you pasted in the same order as printed by company-diag? If yes, then company-lsp is providing the candidates to company-mode in this exact order. I suspect if company-mode itself sorts the candidates somewhere else.

Note that company-lsp explicitly tells company-mode do not sort the candidates again:

company-lsp/company-lsp.el

Line 434 in dd7610a

(sorted t)

And it uses lsp--sort-candidates internally which should sort by sortText:
https://github.com/emacs-lsp/lsp-mode/blob/3a7ccc064cd370131d00ff942ca9e76e8f039a53/lsp-methods.el#L1421-L1425

[ddovod]

Yep, it is first 5 items our of 100, order is preserved.
If so, looks like company does something under the hood, which is weird

[ddovod]

You're right, this piece of code from my emacs config was excess (and probably outdated):

;; Add yasnippet support for all company backends
;; https://github.com/syl20bnr/spacemacs/pull/179
(defvar company-mode/enable-yas t
  "Enable yasnippet for all backends.")

(defun company-mode/backend-with-yas (backend)
  "(as BACKEND) Yasnippet support for company backends."
  (if (or (not company-mode/enable-yas) (and (listp backend) (member 'company-yasnippet backend)))
      backend
    (append (if (consp backend) backend (list backend))
            '(:with company-yasnippet))))

(setq company-backends (mapcar #'company-mode/backend-with-yas company-backends))

Without it company-lsp started to work like a charm!
Thank you very much! Closing the issue