{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":128492826,"defaultBranch":"master","name":"psgd_torch","ownerLogin":"lixilinx","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2018-04-07T03:28:46.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/37510137?v=4","public":true,"private":false,"isOrgOwned":false},"refInfo":{"name":"","listCacheKey":"v0:1711316778.0","currentOid":""},"activityList":{"items":[{"before":"94e39c87c820fe34adc02edf482855d78219127b","after":"3e5ec93db55a03d6b2160ffbd15f6b771eecbe4b","ref":"refs/heads/master","pushedAt":"2024-08-18T03:19:59.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"fixed broken TeX; add ViT demo\n\ngithub tex broken. Better to follow \r\n\r\nhttps://www.onemathematicalcat.org/MathJaxDocumentation/TeXSyntax.htm","shortMessageHtmlLink":"fixed broken TeX; add ViT demo"}},{"before":"06e1ee9c0f3db131fd4e7a3beff5df6f2903a409","after":"94e39c87c820fe34adc02edf482855d78219127b","ref":"refs/heads/master","pushedAt":"2024-08-17T22:42:31.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"a tiny ViT model for adam vs psgd","shortMessageHtmlLink":"a tiny ViT model for adam vs psgd"}},{"before":"6d9ea0f58ce963dfef4d5839671ffbb0f2f89b4d","after":"06e1ee9c0f3db131fd4e7a3beff5df6f2903a409","ref":"refs/heads/master","pushedAt":"2024-08-09T00:24:11.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"necessary updates due to reverting triu01 to triu","shortMessageHtmlLink":"necessary updates due to reverting triu01 to triu"}},{"before":"1af85ad0818053c5abfd15b2408f8366ccaa0de6","after":"6d9ea0f58ce963dfef4d5839671ffbb0f2f89b4d","ref":"refs/heads/master","pushedAt":"2024-08-09T00:21:25.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"revert triu01 back to triu\n\ntriu01 is used to approximate the R of the QR decomposition of I + A as I + triu(A) + triu(A, 1). This causes regression when lr_preconditioner is large, which implies that A is large.","shortMessageHtmlLink":"revert triu01 back to triu"}},{"before":"626b86eea2ed73c74449720ea52f31c3ac628b50","after":"1af85ad0818053c5abfd15b2408f8366ccaa0de6","ref":"refs/heads/master","pushedAt":"2024-08-04T21:15:23.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"add notes for comparison with Adafactor","shortMessageHtmlLink":"add notes for comparison with Adafactor"}},{"before":"0d7f5e567402c6a0a0d74a0001d00d924196318a","after":"626b86eea2ed73c74449720ea52f31c3ac628b50","ref":"refs/heads/master","pushedAt":"2024-08-02T23:34:25.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"PSGD works as expected, but Adafactor not\n\nBoth preconditioners have the same sublinear memory complexity. But unlike PSGD Affine kron(diag, diag), Adafactor is not guaranteed to normalize preconditioned gradients to unit scale (can be arbitrarily bad).","shortMessageHtmlLink":"PSGD works as expected, but Adafactor not"}},{"before":"01d9a34bc8f9bf4ac981c26c31a4c94de5016628","after":"0d7f5e567402c6a0a0d74a0001d00d924196318a","ref":"refs/heads/master","pushedAt":"2024-07-28T19:54:44.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"Cache LU factors for solver in LRA preconditioner \n\nSaved one LU decomposition for the solver in LRA preconditioner","shortMessageHtmlLink":"Cache LU factors for solver in LRA preconditioner"}},{"before":"b72a84c42963fc81132966fd772930276ff66c78","after":"01d9a34bc8f9bf4ac981c26c31a4c94de5016628","ref":"refs/heads/master","pushedAt":"2024-07-22T03:16:58.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"better clarify the role of v for gradient whitening preconditioners","shortMessageHtmlLink":"better clarify the role of v for gradient whitening preconditioners"}},{"before":"4431c67f3e8a59c75185e6714115a026cf5527a2","after":"b72a84c42963fc81132966fd772930276ff66c78","ref":"refs/heads/master","pushedAt":"2024-07-14T04:28:04.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"drop v in affine gradient whitening preconditioner","shortMessageHtmlLink":"drop v in affine gradient whitening preconditioner"}},{"before":"9b1b6dfeb9bf1c2ff637227a04cbfafd70c53e39","after":"4431c67f3e8a59c75185e6714115a026cf5527a2","ref":"refs/heads/master","pushedAt":"2024-07-14T01:30:55.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"test affine whitening preconditioner without v","shortMessageHtmlLink":"test affine whitening preconditioner without v"}},{"before":"30cacce0827582ac569535b21ba59935e4e488e1","after":"9b1b6dfeb9bf1c2ff637227a04cbfafd70c53e39","ref":"refs/heads/master","pushedAt":"2024-07-14T01:28:41.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"integrate out v for affine whitening preconditioner","shortMessageHtmlLink":"integrate out v for affine whitening preconditioner"}},{"before":"f3352f8e35d51fd9f706d34bf4341360bddc0368","after":"30cacce0827582ac569535b21ba59935e4e488e1","ref":"refs/heads/master","pushedAt":"2024-07-08T00:45:18.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"Now affine (kron) support non-2D tensors","shortMessageHtmlLink":"Now affine (kron) support non-2D tensors"}},{"before":"a5da867ecf557978b26546481e55978500a9bfc2","after":"f3352f8e35d51fd9f706d34bf4341360bddc0368","ref":"refs/heads/master","pushedAt":"2024-07-07T20:35:50.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"Update affine (kron) preconditioner to make it work for any tensors","shortMessageHtmlLink":"Update affine (kron) preconditioner to make it work for any tensors"}},{"before":"6de84e6dcd6f610cafdc4007ed7a3a2205232acf","after":"a5da867ecf557978b26546481e55978500a9bfc2","ref":"refs/heads/master","pushedAt":"2024-03-28T16:24:03.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"8aeaf4812585095b22d9c9a1526e0ee20995226e","after":"6de84e6dcd6f610cafdc4007ed7a3a2205232acf","ref":"refs/heads/master","pushedAt":"2024-03-28T15:45:11.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"Update hello_psgd.py","shortMessageHtmlLink":"Update hello_psgd.py"}},{"before":"851635eb29077086f414868fb8b4444052835282","after":"8aeaf4812585095b22d9c9a1526e0ee20995226e","ref":"refs/heads/master","pushedAt":"2024-03-28T15:40:20.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"Update preconditioned_stochastic_gradient_descent.py\n\nThree choices for update invQ with Woodbury identity. The 3rd one seems have less accumulated numerical errors.","shortMessageHtmlLink":"Update preconditioned_stochastic_gradient_descent.py"}},{"before":"f76b5abbd2c93bf60ede4d8c25d2121e60e01b99","after":"851635eb29077086f414868fb8b4444052835282","ref":"refs/heads/master","pushedAt":"2024-03-25T00:18:32.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"c72d5f45cbdb257d0cb160953393c37cbf825110","after":"f76b5abbd2c93bf60ede4d8c25d2121e60e01b99","ref":"refs/heads/master","pushedAt":"2024-03-24T21:46:18.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"add psgd vs shampoo","shortMessageHtmlLink":"add psgd vs shampoo"}},{"before":"75015a7b7fd54c73ef2770867765e6c325e34f6d","after":"c72d5f45cbdb257d0cb160953393c37cbf825110","ref":"refs/heads/master","pushedAt":"2024-03-09T19:01:33.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"b2ca4b476a2f61ec9efd0eeeb9574ada15fbdb98","after":"75015a7b7fd54c73ef2770867765e6c325e34f6d","ref":"refs/heads/master","pushedAt":"2024-03-09T18:06:37.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"add comparison with shampoo","shortMessageHtmlLink":"add comparison with shampoo"}},{"before":"b889fff74ee049b5e3dd2f92f47ec62332603261","after":"b2ca4b476a2f61ec9efd0eeeb9574ada15fbdb98","ref":"refs/heads/master","pushedAt":"2024-03-03T23:23:57.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"typo fix","shortMessageHtmlLink":"typo fix"}},{"before":"5a8c770c6ed89ac5bdf94023362ee2465ac1a58d","after":"b889fff74ee049b5e3dd2f92f47ec62332603261","ref":"refs/heads/master","pushedAt":"2024-03-03T22:16:02.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"Delete misc/quadratic_convergence.svg","shortMessageHtmlLink":"Delete misc/quadratic_convergence.svg"}},{"before":"9f0fc09b6d6bf357b73d7e76549473715bd54185","after":"5a8c770c6ed89ac5bdf94023362ee2465ac1a58d","ref":"refs/heads/master","pushedAt":"2024-03-03T22:15:30.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"Delete misc/psgd_vs_bfgs.svg","shortMessageHtmlLink":"Delete misc/psgd_vs_bfgs.svg"}},{"before":"7b99763b45022e6a42a2a26e83346ae44047868f","after":"9f0fc09b6d6bf357b73d7e76549473715bd54185","ref":"refs/heads/master","pushedAt":"2024-03-03T22:14:41.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"Delete misc/cifar10_resnet.py","shortMessageHtmlLink":"Delete misc/cifar10_resnet.py"}},{"before":"82264ca38cf38d80238a7c186956c8a91d187f2e","after":"7b99763b45022e6a42a2a26e83346ae44047868f","ref":"refs/heads/master","pushedAt":"2024-03-03T22:13:32.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"caeb02554c8b62132b31bb38901da80ec2bb1a1f","after":"82264ca38cf38d80238a7c186956c8a91d187f2e","ref":"refs/heads/master","pushedAt":"2024-03-03T17:24:58.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"psgd lib update","shortMessageHtmlLink":"psgd lib update"}},{"before":"4ae402de6761e3489b69ea0eb43e6d0bea12a922","after":"caeb02554c8b62132b31bb38901da80ec2bb1a1f","ref":"refs/heads/master","pushedAt":"2024-03-03T17:23:37.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"psgd lib update","shortMessageHtmlLink":"psgd lib update"}},{"before":"a153277edca75932fd57e155fee0718a717b067e","after":"4ae402de6761e3489b69ea0eb43e6d0bea12a922","ref":"refs/heads/master","pushedAt":"2024-01-21T19:32:50.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"better readme","shortMessageHtmlLink":"better readme"}},{"before":"22b2e021667996eb0f6133505bdb413f7c9245d7","after":"a153277edca75932fd57e155fee0718a717b067e","ref":"refs/heads/master","pushedAt":"2024-01-21T05:52:39.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"better readme","shortMessageHtmlLink":"better readme"}},{"before":"21bc4a69f7af510617b90bda4eadb53654d2999f","after":"22b2e021667996eb0f6133505bdb413f7c9245d7","ref":"refs/heads/master","pushedAt":"2024-01-15T17:33:37.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"lixilinx","name":null,"path":"/lixilinx","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37510137?s=80&v=4"},"commit":{"message":"multi-runs for tensor rank decomposition benchmark","shortMessageHtmlLink":"multi-runs for tensor rank decomposition benchmark"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEnRfljQA","startCursor":null,"endCursor":null}},"title":"Activity · lixilinx/psgd_torch"}