Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Workspace][Feature] Import sample data to workspace #210

Merged
merged 3 commits into from
Oct 13, 2023
Merged

[Workspace][Feature] Import sample data to workspace #210

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0a17456
feat: import sample data saved objects to workspace
wanglam Aug 31, 2023
c0abc45
refactor: simplify sample data saved object id prefix logic (#1)
ruanyl Sep 12, 2023
28732c5
refactor: assigned copied saved objects to new variables
wanglam Oct 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions src/plugins/home/public/application/components/sample_data_set_cards.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,10 @@ export class SampleDataSetCards extends React.Component {
loadSampleDataSets = async (dataSourceId) => {
let sampleDataSets;
try {
sampleDataSets = await listSampleDataSets(dataSourceId);
sampleDataSets = await listSampleDataSets(
dataSourceId,
getServices().workspaces.currentWorkspaceId$.getValue()
);
} catch (fetchError) {
this.toastNotifications.addDanger({
title: i18n.translate('home.sampleDataSet.unableToLoadListErrorMessage', {
Expand Down Expand Up @@ -114,7 +117,12 @@ export class SampleDataSetCards extends React.Component {
}));

try {
await installSampleDataSet(id, targetSampleDataSet.defaultIndex, dataSourceId);
await installSampleDataSet(
id,
targetSampleDataSet.defaultIndex,
dataSourceId,
getServices().workspaces.currentWorkspaceId$.getValue()
);
} catch (fetchError) {
if (this._isMounted) {
this.setState((prevState) => ({
Expand Down Expand Up @@ -162,7 +170,12 @@ export class SampleDataSetCards extends React.Component {
}));

try {
await uninstallSampleDataSet(id, targetSampleDataSet.defaultIndex, dataSourceId);
await uninstallSampleDataSet(
id,
targetSampleDataSet.defaultIndex,
dataSourceId,
getServices().workspaces.currentWorkspaceId$.getValue()
);
} catch (fetchError) {
if (this._isMounted) {
this.setState((prevState) => ({
Expand Down
2 changes: 2 additions & 0 deletions src/plugins/home/public/application/opensearch_dashboards_services.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ import {
SavedObjectsClientContract,
IUiSettingsClient,
ApplicationStart,
WorkspaceStart,
} from 'opensearch-dashboards/public';
import { UiStatsMetricType } from '@osd/analytics';
import { TelemetryPluginStart } from '../../../telemetry/public';
Expand Down Expand Up @@ -73,6 +74,7 @@ export interface HomeOpenSearchDashboardsServices {
getBranding: () => HomePluginBranding;
};
dataSource?: DataSourcePluginStart;
workspaces: WorkspaceStart;
}

let services: HomeOpenSearchDashboardsServices | null = null;
Expand Down
23 changes: 16 additions & 7 deletions src/plugins/home/public/application/sample_data_client.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,13 +36,13 @@ function clearIndexPatternsCache() {
getServices().indexPatternService.clearCache();
}

export async function listSampleDataSets(dataSourceId) {
const query = buildQuery(dataSourceId);
export async function listSampleDataSets(dataSourceId, workspaceId) {
const query = buildQuery(dataSourceId, workspaceId);
return await getServices().http.get(sampleDataUrl, { query });
}

export async function installSampleDataSet(id, sampleDataDefaultIndex, dataSourceId) {
const query = buildQuery(dataSourceId);
export async function installSampleDataSet(id, sampleDataDefaultIndex, dataSourceId, workspaceId) {
const query = buildQuery(dataSourceId, workspaceId);
await getServices().http.post(`${sampleDataUrl}/${id}`, { query });

if (getServices().uiSettings.isDefault('defaultIndex')) {
Expand All @@ -52,8 +52,13 @@ export async function installSampleDataSet(id, sampleDataDefaultIndex, dataSourc
clearIndexPatternsCache();
}

export async function uninstallSampleDataSet(id, sampleDataDefaultIndex, dataSourceId) {
const query = buildQuery(dataSourceId);
export async function uninstallSampleDataSet(
id,
sampleDataDefaultIndex,
dataSourceId,
workspaceId
) {
const query = buildQuery(dataSourceId, workspaceId);
await getServices().http.delete(`${sampleDataUrl}/${id}`, { query });

const uiSettings = getServices().uiSettings;
Expand All @@ -68,12 +73,16 @@ export async function uninstallSampleDataSet(id, sampleDataDefaultIndex, dataSou
clearIndexPatternsCache();
}

function buildQuery(dataSourceId) {
function buildQuery(dataSourceId, workspaceId) {
const query = {};

if (dataSourceId) {
query.data_source_id = dataSourceId;
}

if (workspaceId) {
query.workspace_id = workspaceId;
}

return query;
}
1 change: 1 addition & 0 deletions src/plugins/home/public/plugin.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,7 @@ export class HomePublicPlugin
featureCatalogue: this.featuresCatalogueRegistry,
injectedMetadata: coreStart.injectedMetadata,
dataSource,
workspaces: coreStart.workspaces,
});
coreStart.chrome.docTitle.change(
i18n.translate('home.pageTitle', { defaultMessage: 'Home' })
Expand Down
8 changes: 3 additions & 5 deletions src/plugins/home/server/services/sample_data/data_sets/ecommerce/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ import { i18n } from '@osd/i18n';
import { getSavedObjects } from './saved_objects';
import { fieldMappings } from './field_mappings';
import { SampleDatasetSchema, AppLinkSchema } from '../../lib/sample_dataset_registry_types';
import { getSavedObjectsWithDataSource, appendDataSourceId } from '../util';
import { addPrefixTo } from '../util';

const ecommerceName = i18n.translate('home.sampleData.ecommerceSpecTitle', {
defaultMessage: 'Sample eCommerce orders',
Expand All @@ -55,13 +55,11 @@ export const ecommerceSpecProvider = function (): SampleDatasetSchema {
darkPreviewImagePath: '/plugins/home/assets/sample_data_resources/ecommerce/dashboard_dark.png',
hasNewThemeImages: true,
overviewDashboard: DASHBOARD_ID,
getDataSourceIntegratedDashboard: appendDataSourceId(DASHBOARD_ID),
getDashboardWithPrefix: addPrefixTo(DASHBOARD_ID),
appLinks: initialAppLinks,
defaultIndex: DEFAULT_INDEX,
getDataSourceIntegratedDefaultIndex: appendDataSourceId(DEFAULT_INDEX),
getDataSourceIntegratedDefaultIndex: addPrefixTo(DEFAULT_INDEX),
savedObjects: getSavedObjects(),
getDataSourceIntegratedSavedObjects: (dataSourceId?: string, dataSourceTitle?: string) =>
getSavedObjectsWithDataSource(getSavedObjects(), dataSourceId, dataSourceTitle),
dataIndices: [
{
id: 'ecommerce',
Expand Down
8 changes: 3 additions & 5 deletions src/plugins/home/server/services/sample_data/data_sets/flights/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ import { i18n } from '@osd/i18n';
import { getSavedObjects } from './saved_objects';
import { fieldMappings } from './field_mappings';
import { SampleDatasetSchema, AppLinkSchema } from '../../lib/sample_dataset_registry_types';
import { getSavedObjectsWithDataSource, appendDataSourceId } from '../util';
import { addPrefixTo } from '../util';

const flightsName = i18n.translate('home.sampleData.flightsSpecTitle', {
defaultMessage: 'Sample flight data',
Expand All @@ -55,13 +55,11 @@ export const flightsSpecProvider = function (): SampleDatasetSchema {
darkPreviewImagePath: '/plugins/home/assets/sample_data_resources/flights/dashboard_dark.png',
hasNewThemeImages: true,
overviewDashboard: DASHBOARD_ID,
getDataSourceIntegratedDashboard: appendDataSourceId(DASHBOARD_ID),
getDashboardWithPrefix: addPrefixTo(DASHBOARD_ID),
appLinks: initialAppLinks,
defaultIndex: DEFAULT_INDEX,
getDataSourceIntegratedDefaultIndex: appendDataSourceId(DEFAULT_INDEX),
getDataSourceIntegratedDefaultIndex: addPrefixTo(DEFAULT_INDEX),
savedObjects: getSavedObjects(),
getDataSourceIntegratedSavedObjects: (dataSourceId?: string, dataSourceTitle?: string) =>
getSavedObjectsWithDataSource(getSavedObjects(), dataSourceId, dataSourceTitle),
dataIndices: [
{
id: 'flights',
Expand Down
8 changes: 3 additions & 5 deletions src/plugins/home/server/services/sample_data/data_sets/logs/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ import { i18n } from '@osd/i18n';
import { getSavedObjects } from './saved_objects';
import { fieldMappings } from './field_mappings';
import { SampleDatasetSchema, AppLinkSchema } from '../../lib/sample_dataset_registry_types';
import { appendDataSourceId, getSavedObjectsWithDataSource } from '../util';
import { addPrefixTo } from '../util';

const logsName = i18n.translate('home.sampleData.logsSpecTitle', {
defaultMessage: 'Sample web logs',
Expand All @@ -55,13 +55,11 @@ export const logsSpecProvider = function (): SampleDatasetSchema {
darkPreviewImagePath: '/plugins/home/assets/sample_data_resources/logs/dashboard_dark.png',
hasNewThemeImages: true,
overviewDashboard: DASHBOARD_ID,
getDataSourceIntegratedDashboard: appendDataSourceId(DASHBOARD_ID),
getDashboardWithPrefix: addPrefixTo(DASHBOARD_ID),
appLinks: initialAppLinks,
defaultIndex: DEFAULT_INDEX,
getDataSourceIntegratedDefaultIndex: appendDataSourceId(DEFAULT_INDEX),
getDataSourceIntegratedDefaultIndex: addPrefixTo(DEFAULT_INDEX),
savedObjects: getSavedObjects(),
getDataSourceIntegratedSavedObjects: (dataSourceId?: string, dataSourceTitle?: string) =>
getSavedObjectsWithDataSource(getSavedObjects(), dataSourceId, dataSourceTitle),
dataIndices: [
{
id: 'logs',
Expand Down
124 changes: 75 additions & 49 deletions src/plugins/home/server/services/sample_data/data_sets/util.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,60 +4,74 @@
*/

import { SavedObject } from 'opensearch-dashboards/server';
import { cloneDeep } from 'lodash';

export const appendDataSourceId = (id: string) => {
return (dataSourceId?: string) => (dataSourceId ? `${dataSourceId}_` + id : id);
const withPrefix = (...args: Array<string | undefined>) => (id: string) => {
const prefix = args.filter(Boolean).join('_');
if (prefix) {
return `${prefix}_${id}`;
}
return id;
};

export const getSavedObjectsWithDataSource = (
saveObjectList: SavedObject[],
dataSourceId?: string,
dataSourceTitle?: string
): SavedObject[] => {
if (dataSourceId) {
return saveObjectList.map((saveObject) => {
saveObject.id = `${dataSourceId}_` + saveObject.id;
// update reference
if (saveObject.type === 'dashboard') {
saveObject.references.map((reference) => {
if (reference.id) {
reference.id = `${dataSourceId}_` + reference.id;
}
});
export const addPrefixTo = (id: string) => (...args: Array<string | undefined>) => {
return withPrefix(...args)(id);
};

const overrideSavedObjectId = (savedObject: SavedObject, idGenerator: (id: string) => string) => {
savedObject.id = idGenerator(savedObject.id);
// update reference
if (savedObject.type === 'dashboard') {
savedObject.references.map((reference) => {
if (reference.id) {
reference.id = idGenerator(reference.id);
}
});
}

// update reference
if (saveObject.type === 'visualization' || saveObject.type === 'search') {
const searchSourceString = saveObject.attributes?.kibanaSavedObjectMeta?.searchSourceJSON;
const visStateString = saveObject.attributes?.visState;
// update reference
if (savedObject.type === 'visualization' || savedObject.type === 'search') {
const searchSourceString = savedObject.attributes?.kibanaSavedObjectMeta?.searchSourceJSON;
const visStateString = savedObject.attributes?.visState;

if (searchSourceString) {
const searchSource = JSON.parse(searchSourceString);
if (searchSource.index) {
searchSource.index = `${dataSourceId}_` + searchSource.index;
saveObject.attributes.kibanaSavedObjectMeta.searchSourceJSON = JSON.stringify(
searchSource
);
}
}
if (searchSourceString) {
const searchSource = JSON.parse(searchSourceString);
if (searchSource.index) {
searchSource.index = idGenerator(searchSource.index);
savedObject.attributes.kibanaSavedObjectMeta.searchSourceJSON = JSON.stringify(
searchSource
);
}
}

if (visStateString) {
const visState = JSON.parse(visStateString);
const controlList = visState.params?.controls;
if (controlList) {
controlList.map((control) => {
if (control.indexPattern) {
control.indexPattern = `${dataSourceId}_` + control.indexPattern;
}
});
if (visStateString) {
const visState = JSON.parse(visStateString);
const controlList = visState.params?.controls;
if (controlList) {
controlList.map((control) => {
if (control.indexPattern) {
control.indexPattern = idGenerator(control.indexPattern);
}
saveObject.attributes.visState = JSON.stringify(visState);
}
});
}
savedObject.attributes.visState = JSON.stringify(visState);
}
}
};

export const getDataSourceIntegratedSavedObjects = (
savedObjectList: SavedObject[],
dataSourceId?: string,
dataSourceTitle?: string
): SavedObject[] => {
savedObjectList = cloneDeep(savedObjectList);
if (dataSourceId) {
return savedObjectList.map((savedObject) => {
overrideSavedObjectId(savedObject, withPrefix(dataSourceId));

// update reference
if (saveObject.type === 'index-pattern') {
saveObject.references = [
if (savedObject.type === 'index-pattern') {
savedObject.references = [
{
id: `${dataSourceId}`,
type: 'data-source',
Expand All @@ -68,17 +82,29 @@ export const getSavedObjectsWithDataSource = (

if (dataSourceTitle) {
if (
saveObject.type === 'dashboard' ||
saveObject.type === 'visualization' ||
saveObject.type === 'search'
savedObject.type === 'dashboard' ||
savedObject.type === 'visualization' ||
savedObject.type === 'search'
) {
saveObject.attributes.title = saveObject.attributes.title + `_${dataSourceTitle}`;
savedObject.attributes.title = savedObject.attributes.title + `_${dataSourceTitle}`;
}
}

return saveObject;
return savedObject;
});
}

return saveObjectList;
return savedObjectList;
};

export const getWorkspaceIntegratedSavedObjects = (
savedObjectList: SavedObject[],
workspaceId?: string
) => {
savedObjectList = cloneDeep(savedObjectList);
Copy link
Collaborator

@SuZhou-Joe SuZhou-Joe Oct 11, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a little bit confused here, should we assign the copied object to a new variable?

Copy link
Collaborator Author

@wanglam wanglam Oct 11, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are too many attributes update work in overrideSavedObjectId. It's all based references, it will take huge effects if we want to use new variables.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess @SuZhou-Joe want to say: shall we do const savedObjectListCopy = cloneDeep(savedObjectList);?


savedObjectList.forEach((savedObject) => {
overrideSavedObjectId(savedObject, withPrefix(workspaceId));
});
return savedObjectList;
};
6 changes: 1 addition & 5 deletions src/plugins/home/server/services/sample_data/lib/sample_dataset_registry_types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ export interface SampleDatasetSchema<T = unknown> {

// saved object id of main dashboard for sample data set
overviewDashboard: string;
getDataSourceIntegratedDashboard: (dataSourceId?: string) => string;
getDashboardWithPrefix: (...args: Array<string | undefined>) => string;
appLinks: AppLinkSchema[];

// saved object id of default index-pattern for sample data set
Expand All @@ -99,10 +99,6 @@ export interface SampleDatasetSchema<T = unknown> {
// OpenSearch Dashboards saved objects (index patter, visualizations, dashboard, ...)
// Should provide a nice demo of OpenSearch Dashboards's functionality with the sample data set
savedObjects: Array<SavedObject<T>>;
getDataSourceIntegratedSavedObjects: (
dataSourceId?: string,
dataSourceTitle?: string
) => Array<SavedObject<T>>;
dataIndices: DataIndexSchema[];
status?: string | undefined;
statusMsg?: unknown;
Expand Down
Loading
Loading