chore/merge upstream 2024112601 #79

onimsha · 2024-11-27T03:31:57Z

prevent trying to submit to jobclient when it can't take any more work (reduces log spam) (prevent trying to submit to jobclient when it can't take any more wor… danswer-ai/danswer#2482)
Bugfix/larger test runner (Bugfix/larger test runner danswer-ai/danswer#2508)
Move code block default language creation to citation processing (Move code block default language creation to citation processing danswer-ai/danswer#2501)
Citations prompt for slack now includes thread history (Citations prompt for slack now includes thread history danswer-ai/danswer#2510)
Non-spotfix deletion of users (Non-spotfix deletion of users danswer-ai/danswer#2499)
Added permission syncing (Added permission syncing danswer-ai/danswer#2340)
Assistants panel rework (Assistants panel rework danswer-ai/danswer#2509)
Asana connector (community originated) (Asana connector (community originated) danswer-ai/danswer#2485)
Added bool to disable chat_session_id check for search_docs for api
Saml account fastapi deletion (Saml account fastapi deletion danswer-ai/danswer#2512)
Route name (Route name danswer-ai/danswer#2520)
Add start date to persona (Add start date to persona danswer-ai/danswer#2407)
align alembic
Access type (Access type danswer-ai/danswer#2523)
try adding build essential (try adding build essential danswer-ai/danswer#2526)
Wrote tests for the chat apis (Wrote tests for the chat apis danswer-ai/danswer#2525)
update default search assistant selection (update default search assistant selection danswer-ai/danswer#2527)
Fix duplicate users from slack / web (Fix duplicate users from slack / web danswer-ai/danswer#2530)
Add additional custom tooling configuration (Add additional custom tooling configuration danswer-ai/danswer#2426)
Updated modals for clarity (Updated modals for clarity danswer-ai/danswer#2529)
Begin distinguishing upsert operations for clarity (Begin distinguishing upsert operations for clarity danswer-ai/danswer#2535)
Add support for o1 (Add support for o1 danswer-ai/danswer#2538)
Support svg navigation items (Support svg navigation items danswer-ai/danswer#2542)
Tooling testing (Tooling testing danswer-ai/danswer#2533)
Fix misc tool call errors (Fix misc tool call errors danswer-ai/danswer#2544)
test self hosted runner (test self hosted runner danswer-ai/danswer#2541)
Feature/runs on 2 (Feature/runs on 2 danswer-ai/danswer#2547)
rename classes and ignore deprecation warnings we mostly don't have c… (rename classes and ignore deprecation warnings we mostly don't have c… danswer-ai/danswer#2546)
fix saml email login upsert issue
Added doc_set__user_group cleanup for user_group deletion (Added doc_set__user_group cleanup for user_group deletion danswer-ai/danswer#2551)
Update theming (Update theming danswer-ai/danswer#2552)
ensure default_assistant passed through
Update merge queue logic (Update merge queue logic danswer-ai/danswer#2554)
Allow = in tag filter (Allow = in tag filter danswer-ai/danswer#2548)
fix typo (fix typo danswer-ai/danswer#2543)
Update some ux edge cases (Update some ux edge cases danswer-ai/danswer#2545)
flex none (Flex none for assistant display danswer-ai/danswer#2558)
ssl_ca_certs should default to None, not "". (ssl_ca_certs should default to None, not "". danswer-ai/danswer#2560)
update folder list (Update folder list danswer-ai/danswer#2563)
Feature/test pruning (Feature/test pruning danswer-ai/danswer#2556)
Cleaned up foreign key cleanup for user group deletion (Cleaned up foreign key cleanup for user group deletion danswer-ai/danswer#2559)
push to docker latest when git tag contains "latest", and tag nightly (push to docker latest when git tag contains "latest", and tag nightly danswer-ai/danswer#2564)
Add open in new tab for custom links (Add open in new tab for custom links danswer-ai/danswer#2568)
clamp retry to max_delay (clamp retry to max_delay danswer-ai/danswer#2570)
Added confluence permission syncing (Added confluence permission syncing danswer-ai/danswer#2537)
Feature/celery multi (Feature/celery multi danswer-ai/danswer#2470)
try user deploy key approach to tagging (try user deploy key approach to tagging danswer-ai/danswer#2575)
Feature/xenforo (Feature/xenforo danswer-ai/danswer#2497)
Minor role display refactor (Minor role display refactor danswer-ai/danswer#2578)
Add size limit to jira tickets (Add size limit to jira tickets danswer-ai/danswer#2586)
Global Curator Fix + Testing (Global Curator Fix + Testing danswer-ai/danswer#2591)
don't write a nightly tag to the same commit more than once (don't write a nightly tag to the same commit more than once danswer-ai/danswer#2585)
Minor Query History Fix (Minor Query History Fix danswer-ai/danswer#2594)
Show transition card if no connectors (Show transition card if no connectors danswer-ai/danswer#2597)
EKS initial deployment (EKS initial deployment danswer-ai/danswer#2154)
Fix default assistant (Fix default assistant danswer-ai/danswer#2600)
prevent nextFormStep unless credential fully set up (Prevent nextFormStep unless credential fully set up danswer-ai/danswer#2599)
Rely on PVC (Rely on PVC danswer-ai/danswer#2604)
Add heartbeat to indexing (Add heartbeat to indexing danswer-ai/danswer#2595)
Added permissions syncing for slack (Added permissions syncing for slack danswer-ai/danswer#2602)
Fix overflow of prompt library table (Fix overflow of prompt library table danswer-ai/danswer#2606)
don't push integration testing docker images (don't push integration testing docker images danswer-ai/danswer#2584)
Unstructured UI (Unstructured UI danswer-ai/danswer#2636)
Update CONTRIBUTING.md
update nav link name (update nav link name danswer-ai/danswer#2643)
run the nightly tag overnight relative to pacific time (run the nightly tag overnight relative to pacific time danswer-ai/danswer#2637)
Added quotes to project name to handle reserved words (Added quotes to project name to handle reserved words danswer-ai/danswer#2639)
welcome flow
raise redis connections and using blocking connection pool for more d… (raise redis connections and using blocking connection pool for more d… danswer-ai/danswer#2635)
add default model + minor fixes (add default model + minor fixes danswer-ai/danswer#2638)
sync up when checks run with branch protection required checks (sync up when checks run with branch protection required checks danswer-ai/danswer#2628)
Redis Cache for KV Store (Redis Cache for KV Store danswer-ai/danswer#2603)
Update litellm to fix bedrock models (Update litellm to fix bedrock models danswer-ai/danswer#2649)
fix typo
Dupe welcome modal logic (Dupe welcome modal logic danswer-ai/danswer#2656)
Clean, memoized assistant ordering (Clean, memoized assistant ordering danswer-ai/danswer#2655)
replace trivy download endpoint due to db download flakiness on their en… (replace trivy download endpoint due to db download flakiness on their en… danswer-ai/danswer#2661)
Better virtualization (Better virtualization danswer-ai/danswer#2653)
add clarity around assistants and names (Add clarity around assistants and names danswer-ai/danswer#2663)
[tiny] adjust user group sync log ([tiny] adjust user group sync log danswer-ai/danswer#2664)
Implement source testing framework + Slack (Implement source testing framework + Slack danswer-ai/danswer#2650)
Unstructured Update Copy (Unstructured Update Copy danswer-ai/danswer#2668)
delete directly via selection instead of making multiple calls to get chunk ids and delete each one (delete directly via selection instead of making multiple calls to get… danswer-ai/danswer#2666)
disable another flaky assert (disable another flaky assert danswer-ai/danswer#2678)
Allow config of background concurrency (Allow config of background concurrency danswer-ai/danswer#2648)
bump celery to 5.5.0b4 (bump celery to 5.5.0b4 danswer-ai/danswer#2681)
update_single should be optimized for a single call now (update_single should be optimized for a single call now danswer-ai/danswer#2671)
harden connections to redis (harden connections to redis danswer-ai/danswer#2677)
Improved api key forms + fix non-submittable azure (Improved api key forms + fix non-submittable azure danswer-ai/danswer#2654)
Add only multi tenant dependency injection (Add only multi tenant dependency injection danswer-ai/danswer#2588)
add multi tenant alembic (Multi tenant alembic danswer-ai/danswer#2589)
cors update (docker deployment cors update danswer-ai/danswer#2686)
Add option to adjust pool size (Add option to adjust pool size danswer-ai/danswer#2695)
Tenant provisioning in the dataplane (Tenant provisioning in the dataplane danswer-ai/danswer#2694)
disabled llm when skip_gen_ai_answer_question set (disabled llm when skip_gen_ai_answer_question set danswer-ai/danswer#2687)
Ensure tests don't use LLM (Ensure tests don't use LLM danswer-ai/danswer#2702)
Enable removal of reranking + navigate back to search settings (Enable removal of reranking + navigate back to search settings danswer-ai/danswer#2674)
remove left-over memo (remove left-over memo danswer-ai/danswer#2669)
update disabling logic ((minor) Update disabling logic danswer-ai/danswer#2592)
DanswerBot Update (DanswerBot Update danswer-ai/danswer#2697)
UX Cleanup (UX Cleanup danswer-ai/danswer#2701)
Feature/background prune 2 (Feature/background prune 2 danswer-ai/danswer#2583)
back button + popups (back button + popups danswer-ai/danswer#2707)
Linting (Linting danswer-ai/danswer#2704)
Minor update to initial modal (Minor update to initial modal danswer-ai/danswer#2571)
Remove Perm Sync Script Dev (Remove Perm Sync Script Dev danswer-ai/danswer#2712)
rely on stdout redirection for supervisord logging (rely on stdout redirection for supervisord logging danswer-ai/danswer#2711)
check before using fetch_versioned_implementation because it logs warnings that confuse users. (check before using fetch_versioned_implementation because it logs war… danswer-ai/danswer#2708)
Inline code + effect clarity (Inline code + effect clarity danswer-ai/danswer#2715)
add one more retry and wait a little longer to allow ourselves to recover from infra issues (add one more retry and wait a little longer to allow ourselves to rec… danswer-ai/danswer#2714)
better logging for actions being taken inside document_by_cc_pair_cleanup (better logging for actions being taken inside document_by_cc_pair_cle… danswer-ai/danswer#2713)
Fix all LegacyKeyValueFormat docker warnings
ensure all timeout -> hook (ensure all timeout -> hook danswer-ai/danswer#2718)
k ((minor) catch leftover dependency fix danswer-ai/danswer#2721)
Temp patch to remove multiple tool calls (Temp patch to remove multiple tool calls danswer-ai/danswer#2720)
experimental workflow to auto merge hotfixes to release branches.
Better support for image generation capable models (Better support for image generation capable models danswer-ai/danswer#2725)
try porting docker web build to runs-on
disable flaky test
urlencode the password part properly before putting it in the broker url (urlencode redis passwords when creating url's for celery danswer-ai/danswer#2719)
Handle bug with initial connector page display (Handle bug with initial connector page display danswer-ai/danswer#2727)
remove more flaky tests
experimental workflow to auto merge hotfixes to release branches. (experimental workflow to auto merge hotfixes to release branches. danswer-ai/danswer#2723)
Fix azure (Fix azure danswer-ai/danswer#2665)
More robust edge detection (More robust edge detection danswer-ai/danswer#2710)
Remove mildly annoying groups fetch (Remove mildly annoying groups fetch danswer-ai/danswer#2733)
add vespa + embedding timeout env variables (add vespa + embedding timeout env variables danswer-ai/danswer#2689)
update hotfix to use commas
rename
another cut
handle merge commits during cherry picking
need git user
add deploy key
gh cli needs its token
edit step name
fix where GH_TOKEN is set
try capturing gh output and parsing
try to update token permissions
fix pr merge command
add branching
Add sync status (Add sync status danswer-ai/danswer#2743)
print various celery queue lengths (print various celery queue lengths danswer-ai/danswer#2729)
google drive permission sync cleanup (google drive permission sync cleanup danswer-ai/danswer#2749)
Convert images to jpeg (Convert images to jpeg danswer-ai/danswer#2737)
update indexing and slack bot to use stdout options (update indexing and slack bot to use stdout options danswer-ai/danswer#2752)
ensure shared assistants accessible via query params (Ensure shared assistants accessible via query params danswer-ai/danswer#2740)
Fix slack bot follow up questions (Fix slack responses danswer-ai/danswer#2756)
Add tenant context (Add tenant context danswer-ai/danswer#2596)
PDF Encrypted Case (PDF Encrypted Case danswer-ai/danswer#2764)
Typo Fix (Typo Fix danswer-ai/danswer#2766)
Improved logging and added comments (Improved logging and added comments danswer-ai/danswer#2763)
Add cql support for confluence connector (Add cql support for confluence connector danswer-ai/danswer#2679)
account for no visible assistants (account for no visible assistants danswer-ai/danswer#2765)
Fix indexing model server port for warmup (Fix indexing model server port for warmup danswer-ai/danswer#2767)
fix typo (fix typo danswer-ai/danswer#2768)
fix web connector (Fix web connector danswer-ai/danswer#2769)
k (Indexing back button functional danswer-ai/danswer#2771)
k (Ensure scroll works with pasting danswer-ai/danswer#2772)
update several github actions to silence github deprecation warnings (update several github actions to silence github deprecation warnings danswer-ai/danswer#2730)
Multi tenant vespa (Multi tenant vespa danswer-ai/danswer#2762)
slightly cleaner loading (slightly cleaner loading danswer-ai/danswer#2776)
Fix tag background
Fix parallel tool calls (Fix parallel tool calls danswer-ai/danswer#2779)
Mypy (Mypy danswer-ai/danswer#2785)
Update README.md
nit (Ensure we still log indexing url danswer-ai/danswer#2787)
Add cursor to cql confluence (Add cursor to cql confluence danswer-ai/danswer#2775)
Notion Connector Improvements (Notion Connector Improvements danswer-ai/danswer#2789)
Fix stopping of indexing runs when pausing a connector
Handle 'cannotExportFile' + fix forms
nit (Remove dev update danswer-ai/danswer#2795)
Confluence permission sync fix for server deployment (Confluence permission sync fix for server deployment danswer-ai/danswer#2784)
Fix file too large error (Fix file too large error danswer-ai/danswer#2799)
Bugfix/mediawiki (Bugfix/mediawiki danswer-ai/danswer#2800)
Ensure warmup occurs once (Ensure warmup occurs once danswer-ai/danswer#2777)
Rate limiting confluence through redis (Rate limiting confluence through redis danswer-ai/danswer#2798)
fix index attempt refreshing automatically (fix index attempt refreshing automatically danswer-ai/danswer#2791)
Ensure regenerate has dropdown too (Ensure regenerate has dropdown too danswer-ai/danswer#2797)
ensure shared chats are shared (ensure shared chats are shared danswer-ai/danswer#2801)
add ca option (add ca option danswer-ai/danswer#2774)
various multi tenant improvements (various multi tenant improvements danswer-ai/danswer#2803)
improved code block copying (improved code block copying danswer-ai/danswer#2802)
Simpler azure embedding (Simpler azure embedding danswer-ai/danswer#2751)
Notion Empty Property Fix (Notion Empty Property Fix danswer-ai/danswer#2817)
Add custom tool headers (Add custom tool headers danswer-ai/danswer#2773)
Notion Recurse Empty Final Field (Notion Recurse Empty Final Field danswer-ai/danswer#2819)
add llama 3.2 (add llama 3.2 danswer-ai/danswer#2812)
Parsing (Parsing danswer-ai/danswer#2734)
Redirect with query param (Redirect with query param danswer-ai/danswer#2811)
check last_pruned instead of is_pruning (check last_pruned instead of is_pruning danswer-ai/danswer#2748)
ensure assistant response parsed correctly (Ensure assistant response parsed correctly danswer-ai/danswer#2823)
Session id: int -> UUID (Session id: int -> UUID danswer-ai/danswer#2814)
Add support for passthrough auth for custom tool calls (Add support for passthrough auth for custom tool calls danswer-ai/danswer#2824)
Added logging for when a member has no email or username
rearrange logging
prevent alembic from configuring logger (prevent alembic from configuring logger danswer-ai/danswer#2826)
bump litellm (bump litellm danswer-ai/danswer#2827)
Make flakey test still run but not fail CI
simplified the document search function
py
Bugfix/monitor exceptions (Bugfix/monitor exceptions danswer-ai/danswer#2830)
fix sync to use update_single (fix sync to use update_single danswer-ai/danswer#2822)
lock to avoid rare serializable errors (lock to avoid rare serializable errors danswer-ai/danswer#2818)
dont fail flaky tests
add slack markdown formatting (add slack markdown formatting danswer-ai/danswer#2829)
Push multi tenancy for slackbot (Push multi tenancy for slackbot danswer-ai/danswer#2828)
always finalize the serialized transaction so that it doesn't leak ou… (always finalize the serialized transaction so that it doesn't leak ou… danswer-ai/danswer#2843)
add sentry (add sentry danswer-ai/danswer#2786)
multitenant setup (multitenant setup danswer-ai/danswer#2845)
Add all-tenants migration for K8 job (Add all-tenants migration for K8 job danswer-ai/danswer#2846)
no serializable, use with_for_update to lock the row.
Revert "no serializable, use with_for_update to lock the row."
use native rate limiting in the confluence client (use native rate limiting in the confluence client danswer-ai/danswer#2837)
Refactor + add global timeout env variable (Refactor + add global timeout env variable danswer-ai/danswer#2844)
Cleanup connector form (Cleanup connector form danswer-ai/danswer#2849)
use with for update instead of serializable (use with for update instead of serializable danswer-ai/danswer#2848)
fresh indexing feature branch (fresh indexing feature branch danswer-ai/danswer#2790)
Add assistant notifications + update assistant context (Add assistant notifications + update assistant context danswer-ai/danswer#2816)
Empty embedding fix (Empty embedding fix danswer-ai/danswer#2853)
Add errors to search (Add errors to search danswer-ai/danswer#2854)
modify default (Update default tag version danswer-ai/danswer#2856)
(minor) search memoization + context ((minor) search memoization + context danswer-ai/danswer#2732)
update google sites + formik (update google sites + formik danswer-ai/danswer#2834)
missing tenant_id as optional param (missing tenant_id as optional param danswer-ai/danswer#2851)
push vespa managed service configs (push vespa managed service configs danswer-ai/danswer#2857)
ensure all sharp-related packages installed (Minor- standalone output build danswer-ai/danswer#2855)
Docstring Update for Docs (Docstring Update for Docs danswer-ai/danswer#2863)
(minor) quote overflow ((minor) quote overflow danswer-ai/danswer#2862)
Minor tenant ID improvements (Minor tenant ID improvements danswer-ai/danswer#2850)
Minor vespa standardization (Minor vespa standardization danswer-ai/danswer#2861)
modal onboarding clarity (Modal onboarding clarity danswer-ai/danswer#2780)
Make it so you can update model providers (Make it so you can update model providers danswer-ai/danswer#2866)
Refactored Confluence Connector (Refactored Confluence Connector danswer-ai/danswer#2859)
tighter signaling to prevent indexing cleanup from hitting tasks that are just starting (tighter signaling to prevent indexing cleanup from hitting tasks that are just starting danswer-ai/danswer#2867)
Handle models like openai/bedrock/claude-3.5-... (Handle models like openai/bedrock/claude-3.5-... danswer-ai/danswer#2869)
Update reset_indexes.py
add stale issues and pr's cron
Salesforce Connector Note (Salesforce Connector Note danswer-ai/danswer#2872)
Confluence polish (Confluence polish danswer-ai/danswer#2874)
Notion Fix Nested Properties (Notion Fix Nested Properties danswer-ai/danswer#2877)
Feature/celery refactor (Feature/celery refactor danswer-ai/danswer#2813)
Auth on main (Auth on main danswer-ai/danswer#2878)
Implement freshdesk frontend
add copying for unrecognized languages (add copying for unrecognized languages danswer-ai/danswer#2883)
Fix Freshdesk connector date parsing for UTC timestamps
Tiny confluence fix (Tiny confluence fix danswer-ai/danswer#2885)
sticky credential description (sticky credential description danswer-ai/danswer#2886)
Gating Notifications (Gating Notifications danswer-ai/danswer#2868)
Bugfix/indexing UI (Bugfix/indexing UI danswer-ai/danswer#2879)
add default schema config (add default schema config danswer-ai/danswer#2888)
Temporary fix for empty Google App credentials (Temporary fix for empty Google App credentials danswer-ai/danswer#2892)
Multitenant redis update (Multitenant redis update danswer-ai/danswer#2889)
Remove unnecessary log & Add LoadConnector
Replace html processing library with danswer util
Make ticket fetching a seperate function from processing
Remove 2 month time check & Add time range to fetch and process
Clearer azure models (Clearer azure models danswer-ai/danswer#2898)
working chat feedback dump script (with api addition) (working chat feedback dump script (with api addition) danswer-ai/danswer#2891)
update stale workflow
fix typo
More intuitive search settings interfaces (More intuitive search settings interfaces danswer-ai/danswer#2899)
Add global assistants context (Add global assistants context danswer-ai/danswer#2900)
can't add to primary_worker_locks if it doesn't exist (can't add to primary_worker_locks if it doesn't exist danswer-ai/danswer#2903)
try hiding celery task spam (try hiding celery task spam danswer-ai/danswer#2905)
Seeding (Seeding danswer-ai/danswer#2902)
entry["id"] can apparently be a number, so convert to str
Silence unnecessary debug log (Silence unnecessary debug log danswer-ai/danswer#2908)
Auto Backport Partial (Auto Backport Partial danswer-ai/danswer#2910)
Harmless Backport (Harmless Backport danswer-ai/danswer#2911)
Bugfix/connector deletion lockout (Bugfix/connector deletion lockout danswer-ai/danswer#2901)
Dev Experience (Dev Experience danswer-ai/danswer#2912)
Harmless Backport (Harmless Backport danswer-ai/danswer#2914)
Workflow (Workflow danswer-ai/danswer#2915)
Harmless Backport (Harmless Backport danswer-ai/danswer#2916)
Add strict json mode (Add strict json mode danswer-ai/danswer#2917)
Fix IT fixture ordering
Ensure proper modal fallback (Ensure proper modal fallback danswer-ai/danswer#2906)
Add pagination & Remove req.obj from connectors.tsx
Fix structured outputs (Fix structured outputs danswer-ai/danswer#2923)
Tenant integration tests (Tenant integration tests danswer-ai/danswer#2913)
upgraded claude model strings (upgraded claude model strings danswer-ai/danswer#2876)
check for index swap (check for index swap danswer-ai/danswer#2922)
Fixed polling issue with timezone
add posthog + layout rework (add posthog + layout rework danswer-ai/danswer#2926)
Backport Test (Backport danswer-ai/danswer#2930)
Backport Branch Fix (Backport Branch Fix danswer-ai/danswer#2931)
feat: sentry updates (feat: sentry updates danswer-ai/danswer#2929)
Backport Test (Backport Test danswer-ai/danswer#2935)
Backport Auth (Backport Auth danswer-ai/danswer#2936)
ensure proper sentry silencing (ensure proper sentry silencing danswer-ai/danswer#2934)
Backport Permissions (Backport Permissions danswer-ai/danswer#2937)
Harmless Backport Test (Harmless Backport Test danswer-ai/danswer#2938)
Backport Test (Backport Test danswer-ai/danswer#2939)
Backport Test (Backport Test danswer-ai/danswer#2940)
Backport Correct Branch (Backport Correct Branch danswer-ai/danswer#2941)
Backport Test Final (Backport Test Final danswer-ai/danswer#2942)
remove rt + home-grown sitemap parsing (remove rt + home-grown sitemap parsing danswer-ai/danswer#2933)
Tenant aware JWT strategy (Tenant aware JWT strategy danswer-ai/danswer#2943)
Move user fetching to SS + parallelize some server-side calls (Move user fetching to SS + parallelize some server-side calls danswer-ai/danswer#2932)
Clean up code
Feat: add clean logging for api routes (Feat: add clean logging for api routes danswer-ai/danswer#2928)
Refactor & cleanup code, process tickets in batches
a (Add cloning of message state danswer-ai/danswer#2815)
Onboarding nits (Onboarding nits danswer-ai/danswer#2907)
tenant seeding docs (tenant seeding docs danswer-ai/danswer#2925)
avoid image generation tool confusion
nit
Harmless Backport Test (Backport Test danswer-ai/danswer#2950)
Ensure build args passed to cloud web images (Ensure build args passed to cloud web images danswer-ai/danswer#2947)
Trigger from Workflow (Trigger from Workflow danswer-ai/danswer#2951)
Test Workflow Trigger (Test Workflow Trigger danswer-ai/danswer#2952)
Update docker-build-push-backend-container-on-tag.yml
Backport Tag Test (Backport Tag Test danswer-ai/danswer#2954)
GHA Trigger (GHA Trigger danswer-ai/danswer#2955)
Backport (Backport danswer-ai/danswer#2956)
Backport Test (Backport Test danswer-ai/danswer#2957)
Backport Test
Backport Test (Backport Test danswer-ai/danswer#2958)
Backport No Tag
Notion Child Block Fix (Notion Child Block Fix danswer-ai/danswer#2953)
Backport Richard
Backport Test (Backport Test danswer-ai/danswer#2959)
Add back Backport Tags
Backport Touchup
Touchup (Backport danswer-ai/danswer#2961)
Echo Merge Commit (Echo Merge Commit danswer-ai/danswer#2962)
Backport Test (Backport Test danswer-ai/danswer#2963)
Backport Debugging
Backport Test Conn (Backport Test danswer-ai/danswer#2964)
New Credentials GH
Backport Test (Backport Test danswer-ai/danswer#2966)
Revert
Backport Test (Backport Test danswer-ai/danswer#2967)
Add Conditional
Backport Test 4 (Backport Test 4 danswer-ai/danswer#2968)
k
Push Tag
Backport Test 5 (Backport Test 5 danswer-ai/danswer#2969)
Richard Key
Backport Test 6 (Backport Test 6 danswer-ai/danswer#2970)
Backport Final
Backport Test 7 (Backport Test 7 danswer-ai/danswer#2971)
Clean up & comment fetch_tickets
Bugfix/celery light backoff (Bugfix/celery light backoff danswer-ai/danswer#2880)
Fix document time parsing
Fix per_page count
change test port to 8889 (docker desktop is now using port 8888 which blocks the test from working on mac) (change test port to 8889 (docker desktop is now using port 8888 which… danswer-ai/danswer#2972)
Fix nagging double auth issue (Fix nagging double auth issue danswer-ai/danswer#2960)
Distinguish users in posthog (Distinguish users in posthog danswer-ai/danswer#2965)
Cleaner initial chat screen (Cleaner initial chat screen danswer-ai/danswer#2528)
add srem and sadd to tenant wrapper (add srem and sadd to tenant wrapper danswer-ai/danswer#2973)
add indent to scan_iter (Add indent to scan_iter danswer-ai/danswer#2948)
remove empty directory (remove empty directory danswer-ai/danswer#2977)
missing scard
fail gracefully on provider fetch (fail gracefully on provider fetch danswer-ai/danswer#2981)
add additional configuration options (add additional configuration options danswer-ai/danswer#2980)
fixed label filter (fixed label filter danswer-ai/danswer#2978)
Tiny logging clarity improvement (Tiny logging clarity improvement danswer-ai/danswer#2985)
Temp fix to add retries to get_all_vespa_ids_for_document_id
init sqlalchemy in child process (init sqlalchemy in child process danswer-ai/danswer#2987)
add extra tags to pruning logs (add extra tags to pruning logs danswer-ai/danswer#2994)
add regeneration clarity (add regeneration clarity danswer-ai/danswer#2986)
Add alternative auth header (Add alternative auth header danswer-ai/danswer#2999)
Refactor metadata
Fix typing errors
clarity (Pruning + Refresh Clarity danswer-ai/danswer#3001)
Minor theming (Minor theming danswer-ai/danswer#2993)
Disable Mediawiki Tests (Disable Mediawiki Tests danswer-ai/danswer#3005)
only warmup on index swap (only warmup on index swap danswer-ai/danswer#3003)
Gmail Connector Robustify (Gmail Connector Robustify danswer-ai/danswer#3000)
Feature/indexing logs (Feature/indexing logs danswer-ai/danswer#3002)
Update connector.py
Fix model test action name
Update README.md (chore: Added back to top , contributors section and star history graph danswer-ai/danswer#3011)
remove endpoint (Remove unused endpoint danswer-ai/danswer#3014)
Proper tenant reset (Proper tenant reset danswer-ai/danswer#3015)
Remove License Issues (Remove License Issues danswer-ai/danswer#3013)
Refactored Google Drive Connector + Permission Syncing (Refactored Google Drive Connector + Permission Syncing danswer-ai/danswer#2945)
Fix pywikibot droppings (Fix pywikibot droppings danswer-ai/danswer#2924)
Quicky
Do not count API keys as users (Do not count API keys as users danswer-ai/danswer#3022)
Basic multi tenant api key (Basic multi tenant api key danswer-ai/danswer#3004)
Feat/certificate (Feat/certificate danswer-ai/danswer#2998)
Fix answer with specified doc ids (Fix answer with specified doc ids danswer-ai/danswer#2703)
minor foreign key update (minor foreign key update danswer-ai/danswer#3007)
Tremor -> Shadcn (Tremor -> Shadcn danswer-ai/danswer#2983)
experimental github action to scan licenses
we didn't checkout the code, no trivy ignore
don't scan the os packages
Update connector.py
try repo level scan
try installing npm deps
codel permissions?
manual only for now
try pip-license-checker
fix run key
mypy fixes
more mypy fixes
out of scope
out of scope
Cleaned up connector
renamed variables
made id more unique
ID fix
troll logger statement
Super user (Super user danswer-ai/danswer#2944)
Feature/redis connector refactor (Feature/redis connector refactor danswer-ai/danswer#2992)
minor nit
Upgrade to latest NextJS + switch to turbopack (Upgrade to latest NextJS + switch to turbopack danswer-ai/danswer#3027)
temporary update (Include File 61 temporarily danswer-ai/danswer#3037)
Revert temporary modifications (Revert temporary modifications danswer-ai/danswer#3038)
add validated + reformatted dynamic beat acquisition (add validated + reformatted dynamic beat acquisition danswer-ai/danswer#3006)
slight upgrade to image generation prompts (slight upgrade to image generation prompts danswer-ai/danswer#3036)
add CSV display (add CSV display danswer-ai/danswer#3028)
Tool call per message (Tool call per message danswer-ai/danswer#3025)
Last Message Too Large Logging (Last Message Too Large Logging danswer-ai/danswer#3039)
add POSTGRES_IDLE_SESSIONS_TIMEOUT (add POSTGRES_IDLE_SESSIONS_TIMEOUT danswer-ai/danswer#3019)
Add drive sections (Add drive sections danswer-ai/danswer#3040)
add multiple formats to tools (add multiple formats to tools danswer-ai/danswer#3041)
Misc color clean up (Misc color clean up danswer-ai/danswer#3026)
Fix slack bot form + LLM provider form
gmail refactor + permission syncing (gmail refactor + permission syncing danswer-ai/danswer#3021)
add default api keys for cloud users (add default api keys for cloud users danswer-ai/danswer#3044)
ensure we reset all (Ensure we reset all danswer-ai/danswer#3048)
add to async get session as well (add to async get session as well danswer-ai/danswer#3046)
Improve background token refresh
form
Clean up tooltips (Clean up tooltips danswer-ai/danswer#3047)
ensure props aligned (ensure props aligned danswer-ai/danswer#3050)
update
k
clarity
add sentry to spawned indexing task
limit session scope of index attempt (use id's where appropriate as w… (limit session scope of index attempt (use id's where appropriate as w… danswer-ai/danswer#3049)
Create connector
fix stale indexing tasks being allowed to run after a restart
comment why we need sentry here
need to see vespa url for container debugging
evaluate None to default
Fix errors and cleanup
fix usage report pagination
functioning
update slack
updated cleanup
no typing
update slack
validated approach
minor udpate
remove print
update redirect
update persona defaults (update persona defaults danswer-ai/danswer#3042)
Google Drive Improvements (Google Drive Improvements danswer-ai/danswer#3057)
minor improvements (Certificate improvements danswer-ai/danswer#3081)
Evaluate None to default (Evaluate None to default danswer-ai/danswer#3069)
PG Dev Max Connections (PG Dev Max Connections danswer-ai/danswer#3082)
Clean up connector
wait for db before allowing worker to proceed (reduces error spam on … (wait for db before allowing worker to proceed (reduces error spam on … danswer-ai/danswer#3079)
Fix failure cases
Remove linux
Migrate tenant upgrades to data plane (Migrate tenant upgrades to data plane danswer-ai/danswer#3051)
adjust default postgres schema for slack listener (adjust default postgres schema for slack listener danswer-ai/danswer#3088)
account for malformatted starter messages
File Connector Metadata (File Connector Metadata danswer-ai/danswer#3089)
fix saving docker logs (fix saving docker logs danswer-ai/danswer#3090)
Reenable OpenAI Tokenizer (Reenable OpenAI Tokenizer danswer-ai/danswer#3062)
Remove ee (Remove ee danswer-ai/danswer#3093)
Robustify cloud deployment + include initial KEDA configuration (Robustify cloud deployment + include initial KEDA configuration danswer-ai/danswer#3094)
Silence auth logs (Silence auth logs danswer-ai/danswer#3098)
re-enable helm (re-enable helm danswer-ai/danswer#3053)
New assistants api (New assistants api danswer-ai/danswer#3097)
Cleaner EE fallback for no op (Cleaner EE fallback for no op danswer-ai/danswer#3106)
moderate component fixes (moderate component fixes danswer-ai/danswer#3095)
improved mobile scroll (Improved mobile scroll danswer-ai/danswer#3110)
Support LITELLM_EXTRA_BODY env variable (Support LITELLM_EXTRA_BODY env variable danswer-ai/danswer#3119)
doc sync celery refactor (doc sync celery refactor danswer-ai/danswer#3084)
Add referral source to cloud on data plane (Add referral source to cloud on data plane danswer-ai/danswer#3096)
Fix widget example
clean up logging in light worker (clean up logging in light worker danswer-ai/danswer#3072)
update organization (update organization danswer-ai/danswer#3118)
quick fix for google doc sync
limited role api keys (limited role api keys danswer-ai/danswer#3115)
Added support for page within a page in Confluence (Added support for page within a page in Confluence danswer-ai/danswer#3125)
Make curators able to create permission synced connectors (Make curators able to create permission synced connectors danswer-ai/danswer#3126)
add creator id to cc pair (add creator id to cc pair danswer-ai/danswer#3121)
Updated google copy and added non admin oauth support (Updated google copy and added non admin oauth support danswer-ai/danswer#3120)
use text()
scale indexing sql pool based on concurrency (scale indexing sql pool based on concurrency danswer-ai/danswer#3130)
Cloud improvements (Cloud improvements danswer-ai/danswer#3099)
minor clarity (minor clarity danswer-ai/danswer#3116)
Cohere (Cohere danswer-ai/danswer#3111)
unnecessary python setup
Fix Quotes Prompting (Fix Quotes Prompting danswer-ai/danswer#3137)
minor configuration updates (minor configuration updates danswer-ai/danswer#3134)
Improved tokenizer fallback (Improved tokenizer fallback danswer-ai/danswer#3132)
Bugfix/overlapping connectors (Bugfix/overlapping connectors danswer-ai/danswer#3138)
Add error message on JSON decode error in CustomTool
reworked drive+confluence frontend and implied backend changes (reworked drive+confluence frontend and implied backend changes danswer-ai/danswer#3143)
Feature/playwright tests (Feature/playwright tests danswer-ai/danswer#3129)
Fixed int test (Fixed int test danswer-ai/danswer#3148)
nits + pricing
Made external permissioned users and slack users show diff (Made external permissioned users and slack users show diff danswer-ai/danswer#3147)
Add back employee context injection
Add GTM tracking
Update login page

Description

[Provide a brief description of the changes in this PR]

How Has This Been Tested?

[Describe the tests you ran to verify your changes]

Accepted Risk

[Any know risks or failure modes to point out to reviewers]

Related Issue(s)

[If applicable, link to the issue(s) this PR addresses]

Checklist:

All of the automated tests pass
All PR comments are addressed and marked resolved
If there are migrations, they have been rebased to latest main
If there are new dependencies, they are added to the requirements
If there are new environment variables, they are added to all of the deployment methods
If there are new APIs that don't require auth, they are added to PUBLIC_ENDPOINT_SPECS
Docker images build and basic functionalities work
Author has done a final read through of the PR right before merge

…eature/reset_indexes

* modal fallback * nit * k * k

* Fix structured outputs * Add back rest

* check for index swap * initial bones * kk * k * k: * nit * nit * rebase + update * nit * minior update * k * minor integration test fixes * nit * ensure we build test docker image * remove one space * k * ensure we wipe volumes * remove log * typo * nit * k * k

…ture/reset_indexes

* upgraded model strings * trolled * we do a little trolling * reeeeeee * alembic upgrade * added ignore * bump litellm * k * nit --------- Co-authored-by: pablodanswer <pablo@danswer.ai>

* check for index swap * k * minor * k * nit

* add posthog + layout rework * remove posthog node * nit

* ensure proper sentry silencing * add comments

* remove rt * nit * add minor alembic revision * functional migration * replace usp * k * typing

* add tenantJWTSrategy * nit

…r-ai#2932) * Move user fetching to SS * Cleanup * Add more logging * Small cleanup

* feat: add clean logging for api routes * nit * `MULTI_TENANT` must be shared config * nit

* temporary stash * welcome flow * minor update * k * minor updates to welcome flow

* in progress PoC * working limited user, needs routes to be marked next * make selected endpoint available to limited user role * xfail on test_slack_prune * add comment to sync function --------- Co-authored-by: Richard Kuo <rkuo@rkuo.com>

…#3126) * Make curators able to create permission synced connectors * removed editing permission synced connectors for curators * updated tests to use access type instead of is_public * update copy

* add creator id to cc pair * fix alembic head * show email instead of UUID * safer check on email * make foreign key relationships optional * always allow creator to edit (per hagen) * use primary join * no index_doc_batch spam * try this again --------- Co-authored-by: Richard Kuo <rkuo@rkuo.com>

* Updated google copy and added non admin oauth support * backend update * accounted for oauth * further removed class variables * updated sets

* add improved cloud configuration * fix typing * finalize slackbot improvements * minor update * finalized keda * moderate slackbot switch * update some configs * revert * include reset engine!

* add cohere default * finalize * minor improvement * update * update * update configs * ensure we properly expose name(space) for slackbot * update config * config

use text()

…hon_setup unnecessary python setup

* silence warning * improved fallback logic * k * minor cosmetic update * minor logic update * nit

* fix tenant logging * upsert only new/updated docs, but always upsert document to cc pair relationship * better logging and rough cut at testing

…er-ai#3143) * reworked drive+confluence frontend and implied backend changes * fixed oauth admin tests * fixed service account tests * frontend cleanup * copy change * details! * added key * so good * whoops! * fixed mnore treljsertjoslijt * has issue with boolean form * should be done

* initial PoC * preliminary working config * first cut at chromatic tests * first cut at chromatic tests * fix yaml * fix yaml again * use workingDir * adapt playwright example * remove env * fix working directory * fix more paths * fix dir * add playwright setup * accidentally deleted a step * update test * think we don't need home.png right now * remove unused home.png --------- Co-authored-by: Richard Kuo <rkuo@rkuo.com>

…i#3147) * Made external permissioned users and slack users show diff * finished * Fix typing * k * Fix * k --------- Co-authored-by: Weves <chrisweaver101@gmail.com>

Signed-off-by: Alex Co <alex.tuan@mindvalley.com>

rkuo-danswer and others added 30 commits October 25, 2024 09:35

Merge branch 'patch-1' of https://github.com/Yash-2707/danswer into f…

84d551e

…eature/reset_indexes

Ensure proper modal fallback (danswer-ai#2906)

76415af

* modal fallback * nit * k * k

Add pagination & Remove req.obj from connectors.tsx

a348caa

Fix structured outputs (danswer-ai#2923)

bd63119

* Fix structured outputs * Add back rest

Merge branch 'main' of https://github.com/danswer-ai/danswer into fea…

0ed77aa

…ture/reset_indexes

upgraded claude model strings (danswer-ai#2876)

94edcac

* upgraded model strings * trolled * we do a little trolling * reeeeeee * alembic upgrade * added ignore * bump litellm * k * nit --------- Co-authored-by: pablodanswer <pablo@danswer.ai>

check for index swap (danswer-ai#2922)

5e01d6b

* check for index swap * k * minor * k * nit

Fixed polling issue with timezone

8023caf

add posthog + layout rework (danswer-ai#2926)

9def9f0

* add posthog + layout rework * remove posthog node * nit

Backport Test (danswer-ai#2930)

8c2138a

Backport Branch Fix (danswer-ai#2931)

c0ab86b

feat: sentry updates (danswer-ai#2929)

848d86b

Backport Test (danswer-ai#2935)

ccc68c5

Backport Auth (danswer-ai#2936)

b4bc6d9

ensure proper sentry silencing (danswer-ai#2934)

e8c28e7

* ensure proper sentry silencing * add comments

Backport Permissions (danswer-ai#2937)

80f8d7a

Harmless Backport Test (danswer-ai#2938)

4bcd65e

Backport Test (danswer-ai#2939)

accdd58

Backport Test (danswer-ai#2940)

f6f9112

Backport Correct Branch (danswer-ai#2941)

e6bef57

Backport Test Final (danswer-ai#2942)

aa0f307

remove rt + home-grown sitemap parsing (danswer-ai#2933)

088551a

* remove rt * nit * add minor alembic revision * functional migration * replace usp * k * typing

Tenant aware JWT strategy (danswer-ai#2943)

1261d85

* add tenantJWTSrategy * nit

Move user fetching to SS + parallelize some server-side calls (danswe…

0c2cc74

…r-ai#2932) * Move user fetching to SS * Cleanup * Add more logging * Small cleanup

Clean up code

e5d84ca

Feat: add clean logging for api routes (danswer-ai#2928)

da3c5e3

* feat: add clean logging for api routes * nit * `MULTI_TENANT` must be shared config * nit

Refactor & cleanup code, process tickets in batches

e5494f9

a (danswer-ai#2815)

a1bfa78

Onboarding nits (danswer-ai#2907)

179dc41

* temporary stash * welcome flow * minor update * k * minor updates to welcome flow

rkuo-danswer and others added 28 commits November 13, 2024 16:15

limited role api keys (danswer-ai#3115)

d703e69

* in progress PoC * working limited user, needs routes to be marked next * make selected endpoint available to limited user role * xfail on test_slack_prune * add comment to sync function --------- Co-authored-by: Richard Kuo <rkuo@rkuo.com>

Added support for page within a page in Confluence (danswer-ai#3125)

60471b6

Make curators able to create permission synced connectors (danswer-ai…

a50a394

…#3126) * Make curators able to create permission synced connectors * removed editing permission synced connectors for curators * updated tests to use access type instead of is_public * update copy

Updated google copy and added non admin oauth support (danswer-ai#3120)

485f3f7

* Updated google copy and added non admin oauth support * backend update * accounted for oauth * further removed class variables * updated sets

use text()

65a205d

scale indexing sql pool based on concurrency (danswer-ai#3130)

d68f8d6

Cloud improvements (danswer-ai#3099)

facf1d5

* add improved cloud configuration * fix typing * finalize slackbot improvements * minor update * finalized keda * moderate slackbot switch * update some configs * revert * include reset engine!

minor clarity (danswer-ai#3116)

6e91964

Cohere (danswer-ai#3111)

7c84105

* add cohere default * finalize * minor improvement * update * update * update configs * ensure we properly expose name(space) for slackbot * update config * config

Merge pull request danswer-ai#3131 from danswer-ai/bugfix/session_text

21d5cc4

use text()

unnecessary python setup

54dc1ac

Merge pull request danswer-ai#3135 from danswer-ai/bugfix/helm_ct_pyt…

637b6d9

…hon_setup unnecessary python setup

Fix Quotes Prompting (danswer-ai#3137)

97932dc

minor configuration updates (danswer-ai#3134)

ddff7ec

Improved tokenizer fallback (danswer-ai#3132)

24be13c

* silence warning * improved fallback logic * k * minor cosmetic update * minor logic update * nit

Bugfix/overlapping connectors (danswer-ai#3138)

7015e6f

* fix tenant logging * upsert only new/updated docs, but always upsert document to cc pair relationship * better logging and rough cut at testing

Add error message on JSON decode error in CustomTool

259fc04

Fixed int test (danswer-ai#3148)

618bc02

nits + pricing

521425a

Made external permissioned users and slack users show diff (danswer-a…

954b5b2

…i#3147) * Made external permissioned users and slack users show diff * finished * Fix typing * k * Fix * k --------- Co-authored-by: Weves <chrisweaver101@gmail.com>

resolve conflicts

e4df35e

Signed-off-by: Alex Co <alex.tuan@mindvalley.com>

Add back employee context injection

df6463f

Signed-off-by: Alex Co <alex.tuan@mindvalley.com>

Add GTM tracking

58c030a

Signed-off-by: Alex Co <alex.tuan@mindvalley.com>

Update login page

7806de7

Signed-off-by: Alex Co <alex.tuan@mindvalley.com>

backport search logic

74ac5b3

Signed-off-by: Alex Co <alex.tuan@mindvalley.com>

onimsha merged commit 144a9dd into main Nov 27, 2024
0 of 7 checks passed

onimsha deleted the chore/merge-upstream-2024112601 branch November 27, 2024 04:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore/merge upstream 2024112601 #79

chore/merge upstream 2024112601 #79

onimsha commented Nov 27, 2024

chore/merge upstream 2024112601 #79

chore/merge upstream 2024112601 #79

Conversation

onimsha commented Nov 27, 2024

Description

How Has This Been Tested?

Accepted Risk

Related Issue(s)

Checklist: