Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add API request states features #1496

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
a51ee63
Add API request states features
Terdious Mar 31, 2022
174c990
removed dataForDownsampling for dataRow method
Terdious Apr 1, 2022
6384feb
removed dataForDownsampling for dataRow method
Terdious Apr 1, 2022
0e0738d
Modify tests on error
Terdious Apr 1, 2022
5d1b6ee
Add apiParams, apiParamExample and apiSuccessExample for REST API
Terdious Apr 1, 2022
bd25cf6
Added device feature selector and external_id in response + Added RES…
Terdious Apr 4, 2022
29de8c3
Remove 'optionnal' parameters in doc API REST
Terdious Apr 4, 2022
2c276d5
modification made to the request following discussion with Pierre-Gilles
Terdious Apr 9, 2022
490dc79
Tests
Terdious Apr 10, 2022
56d0a65
Tests
Terdious Apr 10, 2022
fe1062c
Tests
Terdious Apr 10, 2022
c11315a
Merge branch 'master' into RequestAPI-deviceFeaturesStates
Pierre-Gilles Apr 11, 2022
b49a492
Reduced number of states in tests
Terdious Apr 11, 2022
4354851
Reduced number of states in tests
Terdious Apr 11, 2022
7c23a03
Reduced number of states in tests
Terdious Apr 12, 2022
cf62d5f
Reduced number of states in tests
Terdious Apr 12, 2022
1f4b8c5
Merge branch 'GladysAssistant:master' into RequestAPI-deviceFeaturesS…
Terdious May 3, 2022
2c0455d
Merge branch 'master' into RequestAPI-deviceFeaturesStates
Terdious May 4, 2022
461cc27
Merge branch 'master' into RequestAPI-deviceFeaturesStates
Terdious May 14, 2022
d15081a
Merge branch 'master' into RequestAPI-deviceFeaturesStates
Terdious May 15, 2022
765dc8a
fixes to server/lib/device/device.getDeviceFeaturesStates.js
Terdious May 15, 2022
d6ccab4
Correction and simplification of tests
Terdious May 15, 2022
2db8a69
Merge branch 'master' into RequestAPI-deviceFeaturesStates
Terdious May 16, 2022
82e5ff5
Merge branch 'master' into RequestAPI-deviceFeaturesStates
Terdious May 16, 2022
ad05d6b
Merge branch 'master' into RequestAPI-deviceFeaturesStates
Terdious May 20, 2022
9f56bf2
Merge branch 'master' into RequestAPI-deviceFeaturesStates
Terdious May 21, 2022
b48191d
Merge branch 'master' into RequestAPI-deviceFeaturesStates
Terdious May 24, 2022
9547b3d
Merge branch 'master' into RequestAPI-deviceFeaturesStates
Terdious Jun 1, 2022
b72f8ce
Merge branch 'master' into RequestAPI-deviceFeaturesStates
Terdious Jun 22, 2022
357d346
Merge branch 'master' into RequestAPI-deviceFeaturesStates
Terdious Aug 2, 2022
66fad3a
Merge branch 'master' into RequestAPI-deviceFeaturesStates
Terdious Sep 8, 2022
3ece7b8
Merge branch 'master' into RequestAPI-deviceFeaturesStates
Terdious Oct 29, 2022
1af6e66
Merge branch 'master' into RequestAPI-deviceFeaturesStates
Terdious Nov 2, 2022
61ff68b
Add job in test
Terdious Nov 3, 2022
1168537
Add job in test
Terdious Nov 3, 2022
fa436cb
Merge branch 'master' into RequestAPI-deviceFeaturesStates
Terdious Sep 24, 2023
0d72ed6
Merge branch 'master' into RequestAPI-deviceFeaturesStates
Terdious Oct 2, 2023
5d0bbe9
Fixes following note during server tests (pull_request)
Terdious Oct 2, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 78 additions & 0 deletions server/api/controllers/device.controller.js
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,83 @@ module.exports = function DeviceController(gladys) {
res.json(states);
}

/**
* @api {get} /api/v1/device_feature/:device_feature_selector/states getDeviceFeaturesStates
* @apiName getDeviceFeatureStates
* @apiGroup Device
* @apiParam {string} from - Start date in UTC format "yyyy-mm-ddThh:mm:ss:sssZ"
* or "yyyy-mm-dd hh:mm:ss:sss" (GMT time).
* @apiParam {string} [to="now"] - End date in UTC format "yyyy-mm-ddThh:mm:ss:sssZ"
* or "yyyy-mm-dd hh:mm:ss:sss" (GMT time).
* @apiParam {number} [take] - Number of elements to return.
* @apiParam {number} [skip=0] - Number of elements to skip.
* @apiParam {number} [attributes] - Possible values (separated by a comma ',' if several): 'id',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*string

* 'device_feature_id', 'value', 'created_at' and 'updated_at'. Leave empty to have all the columns.
* @apiParamExample {json} Request-Example with take and skip:
* {
* "from": "2022-04-06T07:00:00.000Z",
* "to": "2022-04-06T21:59:59.999Z",
* "take": 3,
* "skip": 5
* }
* @apiSuccessExample {json} Success-Response with take and skip:
* [
* {
* id: 'e1f30d6e-7891-4484-9aa7-2e094b53ed6c',
* device_feature_id: '83c31637-8cb3-4085-b518-dbe2f89b7d0c',
* value: 13,
* created_at: '2022-04-06 09:05:09.127 +00:00',
* updated_at: '2022-04-06 09:05:09.127 +00:00'
* },
* {
* id: '77bb6449-cdd0-4163-9f38-95102e1cbafa',
* device_feature_id: '83c31637-8cb3-4085-b518-dbe2f89b7d0c',
* value: 16,
* created_at: '2022-04-06 09:06:09.146 +00:00',
* updated_at: '2022-04-06 09:06:09.146 +00:00'
* },
* {
* id: '61a87d2a-93e7-4c6c-bd0e-a337803c236c',
* device_feature_id: '83c31637-8cb3-4085-b518-dbe2f89b7d0c',
* value: 108,
* created_at: '2022-04-06 09:07:09.137 +00:00',
* updated_at: '2022-04-06 09:07:09.137 +00:00'
* }
* ]
* @apiParamExample {json} Request-Example with attributes:
* {
* 'from': "2022-04-06 10:00:00.000",
* 'to': "2022-04-09 23:59:00.000",
* 'attributes': "created_at,value,id"
* }
* @apiSuccessExample {json} Success-Response with attributes definitions:
* [
* {
* created_at: '2022-04-06 10:00:09.225 +00:00',
* value: 139,
* id: '30c43c01-0718-40cd-84e0-b975894bd5af'
* },
* {
* created_at: '2022-04-06 10:01:09.219 +00:00',
* value: 140,
* id: 'd1674409-6baf-4658-abf0-070cbc5cbeef',
* },
* {
* created_at: '2022-04-06 10:02:09.166 +00:00',
* value: 141,
* id: '53129293-0b7c-4ced-be7d-7809fa558c96'
* },
* {
* ...
* },
* ... 3878 more items
* ]
*/
async function getDeviceFeaturesStates(req, res) {
const states = await gladys.device.getDeviceFeaturesStates(req.params.device_feature_selector, req.query);
res.json(states);
}

return Object.freeze({
create: asyncMiddleware(create),
get: asyncMiddleware(get),
Expand All @@ -115,5 +192,6 @@ module.exports = function DeviceController(gladys) {
setValue: asyncMiddleware(setValue),
setValueFeature: asyncMiddleware(setValueFeature),
getDeviceFeaturesAggregated: asyncMiddleware(getDeviceFeaturesAggregated),
getDeviceFeaturesStates: asyncMiddleware(getDeviceFeaturesStates),
});
};
4 changes: 4 additions & 0 deletions server/api/routes.js
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,10 @@ function getRoutes(gladys) {
authenticated: true,
controller: deviceController.getDeviceFeaturesAggregated,
},
'get /api/v1/device_feature/:device_feature_selector/states': {
authenticated: true,
controller: deviceController.getDeviceFeaturesStates,
},
// house
'post /api/v1/house': {
authenticated: true,
Expand Down
69 changes: 69 additions & 0 deletions server/lib/device/device.getDeviceFeaturesStates.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
const { Op } = require('sequelize');
const db = require('../../models');
const { NotFoundError } = require('../../utils/coreErrors');
const { Error400 } = require('../../utils/httpErrors');

const DEFAULT_OPTIONS = {
skip: 0,
order_dir: 'ASC',
order_by: 'created_at',
};

/**
* @description Get all features states aggregates.
* @param {string} selector - Device selector.
* @param {object} options - Options of the query.
* @param {string} options.from - Start date in UTC format "yyyy-mm-ddThh:mm:ss:sssZ"
* or "yyyy-mm-dd hh:mm:ss:sss" (GMT time).
* @param {string} [options.to] - End date in UTC format "yyyy-mm-ddThh:mm:ss:sssZ"
* or "yyyy-mm-dd hh:mm:ss:sss" (GMT time).
* @param {number} [options.take] - Number of elements to return.
* @param {number} [options.skip] - Number of elements to skip.
* @param {string} [options.attributes] - Possible values (separated by a comma ',' if several): 'id',
* 'device_feature_id', 'value', 'created_at' and 'updated_at'. Leave empty to have all the columns.
* @returns {Promise<Array>} - Resolve with an array of data.
* @example
* device.getDeviceFeaturesStates('test-device', {from: '2022-03-31T00:00:00.000Z', to: '2022-03-31T23:59:59.999Z',
* take: 100, skip: 10, attributes: 'id,value,created_at'});
*/
async function getDeviceFeaturesStates(selector, options) {
const deviceFeature = this.stateManager.get('deviceFeature', selector);
if (deviceFeature === null) {
throw new NotFoundError('DeviceFeature not found');
}
if (options.from === undefined) {
throw new Error400('Start date missing');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the core, we try to avoid using HTTP errors (specific to the API), and instead use core errors. In that case, I would use the "BadParameters" error in the require('../../utils/coreErrors');

}
const fromDate = new Date(options.from);
// Default end date is now
const toDate = options.to ? new Date(options.to) : new Date();

const optionsWithDefault = { ...DEFAULT_OPTIONS, ...options };

const queryParams = {
raw: true,
where: {
device_feature_id: deviceFeature.id,
created_at: {
[Op.gte]: fromDate,
[Op.lte]: toDate,
},
},
offset: optionsWithDefault.skip,
order: [[optionsWithDefault.order_by, optionsWithDefault.order_dir]],
};

// take is not a default
if (optionsWithDefault.take !== undefined) {
queryParams.limit = optionsWithDefault.take;
}

if (optionsWithDefault.attributes !== undefined) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the default should be to have only those attributes returned:

	{
    		created_at: '2022-04-06 10:00:09.225 +00:00',
    		value: 139,
    		id: '30c43c01-0718-40cd-84e0-b975894bd5af'
    }

(And the user can modify that if needed)

What do you think of that?

(And don't forget to add a test to it)

queryParams.attributes = optionsWithDefault.attributes.split(',');
}
return db.DeviceFeatureState.findAll(queryParams);
}

module.exports = {
getDeviceFeaturesStates,
};
2 changes: 2 additions & 0 deletions server/lib/device/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ const { getBySelector } = require('./device.getBySelector');
const { getDeviceFeaturesAggregates } = require('./device.getDeviceFeaturesAggregates');
const { getDeviceFeaturesAggregatesMulti } = require('./device.getDeviceFeaturesAggregatesMulti');
const { onHourlyDeviceAggregateEvent } = require('./device.onHourlyDeviceAggregateEvent');
const { getDeviceFeaturesStates } = require('./device.getDeviceFeaturesStates');
const { purgeStates } = require('./device.purgeStates');
const { purgeStatesByFeatureId } = require('./device.purgeStatesByFeatureId');
const { poll } = require('./device.poll');
Expand Down Expand Up @@ -94,6 +95,7 @@ DeviceManager.prototype.getBySelector = getBySelector;
DeviceManager.prototype.getDeviceFeaturesAggregates = getDeviceFeaturesAggregates;
DeviceManager.prototype.getDeviceFeaturesAggregatesMulti = getDeviceFeaturesAggregatesMulti;
DeviceManager.prototype.onHourlyDeviceAggregateEvent = onHourlyDeviceAggregateEvent;
DeviceManager.prototype.getDeviceFeaturesStates = getDeviceFeaturesStates;
DeviceManager.prototype.purgeStates = purgeStates;
DeviceManager.prototype.purgeStatesByFeatureId = purgeStatesByFeatureId;
DeviceManager.prototype.poll = poll;
Expand Down
28 changes: 28 additions & 0 deletions server/test/controllers/device/device.controller.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,34 @@ describe('GET /api/v1/device_feature/aggregated_states', () => {
});
});

describe('GET /api/v1/device_feature/:device_feature_selector/states', () => {
Terdious marked this conversation as resolved.
Show resolved Hide resolved
beforeEach(async function BeforeEach() {
this.timeout(1000);
await insertStates(1);
});
it('should get device feature states by selector', async () => {
const now = new Date();
const dateState = `${now.getUTCFullYear()}-${`0${now.getUTCMonth() + 1}`.slice(-2)}-${`0${now.getUTCDate()}`.slice(
-2,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use dayjs everywhere in Gladys, it might be simpler/more readable just to use a simple:

dayjs().format('YYYY-MM-DD')

No?

)}`;

await authenticatedRequest
.get('/api/v1/device_feature/test-device-feature/states')
.query({
from: new Date(`${dateState}T00:00:00.000Z`),
to: new Date(`${dateState}T23:59:59.999Z`),
})
.expect('Content-Type', /json/)
.expect(200)
.then((res) => {
expect(res.body).to.have.lengthOf(2000);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bit heavy to test returning 2000 state. The goal of inserting 2000 states for aggregated tests was to verify that only 100 of the 2000 were returned in the JSON response.

Here, testing that 10, 200, or 2000 are returned doesn't change anything, it'll just slow the test down on every CI run.

I would suggest testing only with like 50-100 states, it should be enough !

expect(res.body).to.be.an('array');
expect(res.body[0]).to.be.an('object');
expect(Object.keys(res.body[0])).to.have.lengthOf(5);
});
});
});

describe('DELETE /api/v1/device/:device_selector', () => {
it('should delete device', async () => {
await authenticatedRequest
Expand Down
91 changes: 91 additions & 0 deletions server/test/lib/device/device.getDeviceFeaturesStates.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
const EventEmitter = require('events');
const { expect, assert } = require('chai');
const uuid = require('uuid');
const { fake } = require('sinon');
const db = require('../../../models');
const Device = require('../../../lib/device');
const Job = require('../../../lib/job');

const event = new EventEmitter();
const job = new Job(event);

const now = new Date('2000-06-15T03:59:00.000Z');
const insertStates = async () => {
const queryInterface = db.sequelize.getQueryInterface();
const deviceFeatureStateToInsert = [];
const statesToInsert = 3 * 60;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is still based from the tests from the aggregated states, but it doesn't seem to be really useful to have that much states ?

IMO, you're doing a simple function, you can write a simple test

In your case, you could even hardcode some states here (like 4 states) with fixed dates.

Then, in your test, you write a query between fixed dates and make sure that only 2 of the 4 states are returned ? (for example)

If you want to test the ?take, you could add another test and do it with ?take=1 for example

for (let i = 0; i < statesToInsert; i += 1) {
const startAt = new Date(now.getTime() - 3 * 60 * 60 * 1000);
const date = new Date(startAt.getTime() + ((3 * 60 * 60 * 1000) / statesToInsert) * i);
deviceFeatureStateToInsert.push({
id: uuid.v4(),
device_feature_id: 'ca91dfdf-55b2-4cf8-a58b-99c0fbf6f5e4',
value: i,
created_at: date,
updated_at: date,
});
}
await queryInterface.bulkInsert('t_device_feature_state', deviceFeatureStateToInsert);
};

describe('Device.getDeviceFeaturesStates', function Describe() {
this.timeout(5000);

afterEach(async () => {
const queryInterface = db.sequelize.getQueryInterface();
await queryInterface.bulkDelete('t_device_feature_state');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not useful, database is automatically purged between each tests

});
it('should return states between 01:10 and 02:09 with a target between 2000-06-15 00:10 and now using take and skip , only created_at and values', async () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test assertion seems to test a very specific use case, and I don't see anything in the test code that checks that it's working as advertised?

await insertStates();
const variable = {
getValue: fake.resolves(null),
};
const stateManager = {
get: fake.returns({
id: 'ca91dfdf-55b2-4cf8-a58b-99c0fbf6f5e4',
name: 'my-feature',
}),
};
const dateState = '2000-06-15';
const device = new Device(event, {}, stateManager, {}, {}, variable, job);
const states = await device.getDeviceFeaturesStates('test-device-feature', {
from: new Date(`${dateState}T00:00:00.000Z`).toISOString(),
attributes: 'created_at,value',
take: 60,
skip: 10,
});
expect(states).to.have.lengthOf(60);
expect(Object.keys(states[0])).to.have.lengthOf(2);
expect(states[0]).to.have.property('value');
expect(states[0]).to.not.have.own.property('updated_at');
});
it('should return error, device feature doesnt exist', async () => {
const variable = {
getValue: fake.resolves(null),
};
const stateManager = {
get: fake.returns(null),
};
const dateState = '2000-06-15';
const device = new Device(event, {}, stateManager, {}, {}, variable, job);
const promise = device.getDeviceFeaturesStates('test-device-feature', {
from: new Date(`${dateState}T00:00:00.000Z`).toISOString(),
to: new Date(`${dateState}T10:00:00.000Z`).toISOString(),
});
return assert.isRejected(promise, 'DeviceFeature not found');
});
it('should return error, start date missing', async () => {
const variable = {
getValue: fake.resolves(null),
};
const stateManager = {
get: fake.returns({
id: 'ca91dfdf-55b2-4cf8-a58b-99c0fbf6f5e4',
name: 'my-feature',
}),
};
const device = new Device(event, {}, stateManager, {}, {}, variable, job);
const promise = device.getDeviceFeaturesStates('this-device-does-not-exist', {});
return assert.isRejected(promise, 'Start date missing');
});
});
Loading