Skip to content

Commit

Permalink
Multi-repo (#1153)
Browse files Browse the repository at this point in the history
* wip: multi-repo

wip: multirepo

wip: multirepo

Use RepoPath in more places

Minor things

Clippy & some small things

Resurrect explain endpoint

Add Project to link references

Clean up call sites to semantic search

Always search on display name

We can propagate multiple repos, so this shouldn't be needed

Scope fuzzy queries to a project

Use RepoRef in RepoPath

Use RepoPath for relative_path refs

Add back sqlx data

Add back branch filters to fuzzy matching

This is just awful

Eh, need correct json here

---------

Co-authored-by: rsdy <p@symmetree.dev>

* update answer prompt for multi-repo

* WIP: projects

* WIP: projects

Back-end API changes include:

- Addition of `projects` table
  - Studios now live *inside* a project
  - Ownership was moved from studios to projects
- New routes:
  - `GET /api/projects`: returns a list of:

    `[ { id: number, name: string, modified_at: date string } ]`

  - `POST /api/projects/`: takes in a body like:

    `{ name: string | null }`

    Note: there is a default name generated here if not provided, "New Project"

    This route returns a string body which is the ID

  - `GET /api/projects/:id`: if the project exists, returns:

    `{ name: string }`

  - `POST /api/projects/:id`: updates a project, takes a body of:

    `{ name: string }`

    Returns nothing

- Additionally, all `/api/studio/...` routes have been moved to
  `/projects/:id/studios/..`
  - Note: `studio` was changed to `studios`
  - Note: all routes remain otherwise unchanged

* WIP: Projects

* WIP: Projects

* WIP: Projects

* WIP: Projects - refactoring conversation threads for multi-project context

* WIP: Projects

As part of this change, we add some new routes for project repo
associations.

- `GET /projects/:id/repos` returns a list of:

  `[ { ref: string } ]`

- `POST /projects/:id/repos`: takes in a body like:

  `{ ref: string }`

  This adds the repo by repo ref to the list of repos in a project

- `DELETE /projects/:id/repos/:repo_ref` deletes the repo from the
  project repo list

* WIP: Projects

As part of this change, we now return a complete `Repo` object when
retrieving `GET /projects/:id/repos`.

Tooling for the agent was also adjusted.

* WIP: Projects

Here, we add `most_common_langs` to `GET /projects` and`GET /projects/:id`

We also add routes for project to doc associations.

- `GET /projects/:id/docs` returns a list of:

  ```
  [
    {
      id: number,
      url: string,
      index_status: string,
      name: null | string,
      favicon: null | string,
      description: null | string,
      modified_at: date string,
    }
  ]
  ```

- `POST /projects/:id/docs`: takes in a body like:

  `{ doc_id: number }`

  This adds the doc by ID to the list of docs in a project

- `DELETE /projects/:id/docs/:doc_id` deletes the doc from the project doc
  list

* WIP: Projects

Add branches to `project_repo` associations

* WIP: Projects

Add constraints and foreign keys on new project models

* WIP: Projects

Amongst other patches, we introduce some API changes here.

- We move `/q` to `/projects/:id/q`:
  - This no longer takes a `repo_ref` argument. Now, this route will
    infer the related repositories based on repos associated with the
    requested project.
- We move `/search/path` to `/projects/:id/search/path`
- We add a new `GET /folder` route, which is like `/file` but retrieves
  directory data. Internally, this route makes an `open:`-style query.

* fix repo tantivy search (#1174)

* Fix conversation store/load

* Add `PUT /projects/:id/repos`, change `DELETE /projects/:id/repos/:id`

We add `PUT /projects/:id/repos`, which accepts an object:

```
{ "ref": repo ref, "branch": branch name or NULL }
```

Additionally, `DELETE /projects/:id/repos/:repo_ref` was changed to just
`DELETE /projects/:id/repos/:id`, where the `repo_ref` value was moved
to a JSON object in the request body:

```
{ "ref": repo ref }
```

* Avoid JSON body in `DELETE /projects/:id/repos`

Now, we use a query parameter to indicate the repo ref: `?ref=...`

* Filter repos in semantic search by query repos, if present

* Sync docs in background, whether stream still exists

* Return thread_id in conversation routes

- `GET /projects/:id/conversations/:id` now returns an object like:

  ```
  { thread_id: string, exchanges: [...] }
  ```

  Note that previously, this just returned the list of exchanges.

- `GET /projects/:id/conversations` now returns an additional field in
  each item:

  ```
  { thread_id: string, ...previous fields }
  ```

* Fix handling of `conversation_id` with `/answer`

* Fix routing for `DELETE /projects/:id/conversations/:id`

* Use `conversation_id` instead of `thread_id` in `GET /answer`

Rather than returning an initial JSON object, we introduce a new
`ChatEvent` type, and return the conversation ID on stream end upon
successful store.

* Return errors with debug formatting

* Fix more rebase errors

* Indexing status reporting improvements (#1192)

* repo index status reporting fixes

* report whether is resync in index progress

* rework sync logic for docs (#1186)

* rework sync logic for docs

- replace `/sync` with `/enqueue`; a non-streaming replacement to add
  items to the doc-sync queue
- introduce `/status` and `/cancel`; to stream updates for a syncing
  document or to cancel a sync job
- convert `/resync` to http from sse
- internal updates to `/list` to work with the new queue system

* track metadata update in progress stream

* handle possible error state

* Fix tests

* Run cargo fmt

* Path search edits (#1200)

* add repo name to path tool answer, use skim_fuzzy_path_match instead of fuzzy_path_match and use only repos from the project

* filter fuzzy path search by language and remove unused code paths

---------

Co-authored-by: rafael <22560219+rmuller-ml@users.noreply.github.com>

* Add repos to answer action prompt and step prompt (#1198)

* Add repos to answer action prompt and step prompt

* limit number of tokens for symbol classification

* tweak prompt text

---------

Co-authored-by: Gabriel Gordon-Hall <ggordonhall@gmail.com>

* Restrict queries on `/q` to only return results valid for a project (#1203)

We rewrite the parsed set of queries to restrict them such that they only return valid results.

* Restrict queries on `/q` to only return results valid for a project

* Add project ID to autocomplete

* Fix repo autocomplete

* fix default for repo deduplication in semantic search

* Fix autocomplete for path and lang queries

* Send back context data in studio list route

* Fix code search on local repos (#1204)

* Fix code search on local repos

We were making semantic queries with the full stringified repo ref.
Instead, we should have been constructing a semantic query using the
repository display name.

It seems that this was fixed coincidentally in #1190 via a condition
(which might now be possible to remove).

* Use indexed name in semantic query construction

* Add fields to list studios (projects) (#1205)

* return token counter, doc_context and full context in list studios route

* make studio prompt multi-repo (#1208)

* save onboarding status on user profile (#1210)

* Anastasiia/autocomplete page size (#1211)

don't override page_size from api_params

* return token counts for studio snapshots (#1212)

* fix clippy (#1213)

* return None if parent commit does not exist (#1214)

* fix SQL query that retrieves a list of docs for project (#1216)

* Fix autocomplete repo match (#1218)

* make repo_name ; make autocomplete and folder queries case insensitive; remove repo.display_name()

* use stringified repo_ref in prompt

* bump version to 0.6.0

* fix blocking status endpoint (#1217)

the status reporting endpoint can drop the lock over the tantivy index
once it has a handle to the progress stream.

* Migrate existing databases to new project schema

* Fix lints

* app redesign

* disable right click in prod mode

* add files to code studio

* project dependant autocomplete

* some fixes

* add selection hint

* studio conversation

* studio conversation design fixes

* studio navigation using left sidebar

* indexing docs, add docs to projects

* some fixes

* add docs to studios

* polishing studios

* usage popover

* add tutorial cards

* search docs

* fixes

* quick fix

* minor fixes

* fix apply diff

* feedback fixes

* feedback fixes 2

* feedback fixes 3

* dedupe studios, pass page_size to autocomplete

* studio response loading state

* studio history

* add repo to project when it finished indexing

* ensure there are no duplicates in file search, fix error handling in chat

* testing fixes

* add missing translations, clean up locales files

* doc fixes

* improve mouse handling in arrow navigation is command bar

* testing fixes

* rework arrow navigation

* rework arrow navigation, use context

* add arrow navigation to all dropdowns

* minor fixes

* improve header wrapping when adding to studio

---------

Co-authored-by: Gabriel Gordon-Hall <ggordonhall@gmail.com>
Co-authored-by: rsdy <p@symmetree.dev>
Co-authored-by: calyptobai <calyptobai@gmail.com>
Co-authored-by: akshay <nerdy@peppe.rs>
Co-authored-by: rafael <22560219+rmuller-ml@users.noreply.github.com>
Co-authored-by: calyptobai <111788964+calyptobai@users.noreply.github.com>
  • Loading branch information
7 people authored Jan 30, 2024
1 parent 2e2cd7b commit 33d6a5c
Show file tree
Hide file tree
Showing 769 changed files with 30,349 additions and 43,682 deletions.
5 changes: 3 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 1 addition & 4 deletions apps/desktop/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,9 @@
<!-- </script>-->
</head>

<body data-theme="default">
<body data-theme="system">
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
<script>
document.body.dataset['theme'] = localStorage.getItem('theme');
</script>
</body>

</html>
2 changes: 1 addition & 1 deletion apps/desktop/src-tauri/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "bloop"
version = "0.5.12"
version = "0.6.0"
description = "Search code. Fast."
authors = ["Bloop AI Developers"]
license = "Apache-2.0"
Expand Down
7 changes: 4 additions & 3 deletions apps/desktop/src-tauri/tauri.conf.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
},
"package": {
"productName": "bloop",
"version": "0.5.12"
"version": "0.6.0"
},
"tauri": {
"allowlist": {
Expand Down Expand Up @@ -103,8 +103,9 @@
"hiddenTitle": true,
"titleBarStyle": "Overlay",
"minHeight": 700,
"minWidth": 1000
"minWidth": 1000,
"fileDropEnabled": false
}
]
}
}
}
73 changes: 50 additions & 23 deletions apps/desktop/src/App.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ import { polling } from '../../../client/src/utils/requestUtils';
import ReportBugModal from '../../../client/src/components/ReportBugModal';
import { UIContext } from '../../../client/src/context/uiContext';
import { DeviceContextProvider } from '../../../client/src/context/providers/DeviceContextProvider';
import { EnvContext } from '../../../client/src/context/envContext';
import TextSearch from './TextSearch';
import SplashScreen from './SplashScreen';

Expand Down Expand Up @@ -155,7 +156,8 @@ function App() {
const handleKeyEvent = useCallback((e: KeyboardEvent) => {
if (
(e.key === '=' || e.key === '-' || e.key === '0') &&
(e.metaKey || e.ctrlKey)
(e.metaKey || e.ctrlKey) &&
!e.shiftKey
) {
const root = document.querySelector(':root');
if (!root) {
Expand Down Expand Up @@ -185,6 +187,19 @@ function App() {
}
}, []);

useEffect(() => {
const onContextMenu = (e: MouseEvent) => {
if (!import.meta.env.DEV) {
e.preventDefault();
}
};
document.addEventListener('contextmenu', onContextMenu);

return () => {
document.removeEventListener('contextmenu', onContextMenu);
};
}, []);

useEffect(() => {
let intervalId: number;
if (!Object.keys(envConfig).length) {
Expand Down Expand Up @@ -226,12 +241,18 @@ function App() {
isRepoManagementAllowed: true,
forceAnalytics: false,
isSelfServe: false,
envConfig,
setEnvConfig,
showNativeMessage: message,
relaunch,
}),
[homeDirectory, indexFolder, os, release, envConfig],
[homeDirectory, indexFolder, os, release],
);

const envContextValue = useMemo(
() => ({
envConfig,
setEnvConfig,
}),
[envConfig],
);

const bugReportContextValue = useMemo(
Expand All @@ -244,26 +265,32 @@ function App() {
);

return (
<LocaleContext.Provider value={localeContextValue}>
<AnimatePresence initial={false}>
{shouldShowSplashScreen && <SplashScreen />}
</AnimatePresence>
{shouldShowSplashScreen && (
<DeviceContextProvider deviceContextValue={deviceContextValue}>
<UIContext.BugReport.Provider value={bugReportContextValue}>
<ReportBugModal errorBoundaryMessage={serverCrashedMessage} />
</UIContext.BugReport.Provider>
</DeviceContextProvider>
)}
<TextSearch contentRoot={contentContainer.current} />
<div ref={contentContainer}>
<BrowserRouter>
{!shouldShowSplashScreen && (
<ClientApp deviceContextValue={deviceContextValue} />
<DeviceContextProvider
deviceContextValue={deviceContextValue}
envConfig={envConfig}
>
<EnvContext.Provider value={envContextValue}>
<LocaleContext.Provider value={localeContextValue}>
<AnimatePresence initial={false}>
{shouldShowSplashScreen && <SplashScreen />}
</AnimatePresence>
{shouldShowSplashScreen && (
<UIContext.BugReport.Provider value={bugReportContextValue}>
<ReportBugModal errorBoundaryMessage={serverCrashedMessage} />
</UIContext.BugReport.Provider>
)}
</BrowserRouter>
</div>
</LocaleContext.Provider>
<TextSearch contentRoot={contentContainer.current} />
<div
ref={contentContainer}
className="w-screen h-screen overflow-hidden"
>
<BrowserRouter>
{!shouldShowSplashScreen && <ClientApp />}
</BrowserRouter>
</div>
</LocaleContext.Provider>
</EnvContext.Provider>
</DeviceContextProvider>
);
}

Expand Down
2 changes: 1 addition & 1 deletion apps/desktop/src/SplashScreen.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ const SplashScreen = ({}: Props) => {
<div className="animate-pulse-slow text-label-title">
<LogoFull />
</div>
<div className="flex gap-3 items-center rounded-lg shadow-high border border-bg-border bg-bg-shade px-4 py-2 absolute bottom-0 left-1/2 transform -translate-x-1/2 translate-y-1/2 body-s z-10 text-label-base">
<div className="flex gap-3 items-center rounded-lg shadow-high border border-bg-border bg-bg-shade px-4 py-2 absolute bottom-0 left-1/2 transform -translate-x-1/2 translate-y-1/2 body-s-b z-10 text-label-base">
<Trans>Loading...</Trans>
</div>
</div>
Expand Down
2 changes: 1 addition & 1 deletion apps/desktop/src/TextSearch.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ const TextSearch = ({
currentResult={currentResult}
setCurrentResult={setCurrentResult}
searchValue={searchValue}
containerClassName="fixed top-[100px] right-[5px]"
containerClassName="fixed top-[100px] right-[5px] w-80"
/>
);
};
Expand Down
2 changes: 1 addition & 1 deletion client/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
<title>bloop</title>
</head>

<body data-theme="default">
<body data-theme="system">
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
Expand Down
2 changes: 1 addition & 1 deletion client/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@bloop/client",
"private": true,
"version": "0.5.12",
"version": "0.6.0",
"scripts": {
"dev": "vite",
"build": "tsc && vite build",
Expand Down
Binary file modified client/public/bloopHeadMascot.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added client/public/bloopHeadMascotLight.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added client/public/stripe_logo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 33d6a5c

Please sign in to comment.