Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start of Sprint 2 #49

Open
wants to merge 69 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
830d2ec
Formatting
sras Nov 4, 2016
a43ee57
Insert records directily with out having to convert to PGW types
sras Nov 4, 2016
45c7a2d
Use the records directly without converting to thier PG counterparts …
sras Nov 4, 2016
19b697b
Fix issue when dealing with enum field with default value
sras Nov 5, 2016
f6a52f0
Add housekeeping fields to Tenant
sras Nov 7, 2016
46ba90a
Add housekeeping fields to remaining models
sras Nov 7, 2016
e7fef1e
Update housekeeping columns without lens
sras Nov 7, 2016
bc4b2df
Fix postgresql/haskell conversion issue and start using Lens
sras Nov 7, 2016
4e25c7c
Made role api to use create_item function that uses lenses to deal wi…
sras Nov 7, 2016
f11a655
Convert more api function to use lenses
sras Nov 8, 2016
bb234be
Fix json generation
sras Nov 8, 2016
785c7d7
Implementing generic update function
sras Nov 8, 2016
f286b54
Generic update function type checks now
sras Nov 8, 2016
4781ca8
Refactor
sras Nov 8, 2016
e16f7a0
Convert everything to use lens
sras Nov 8, 2016
df887ed
Split the generic insert/update function in a separate module
sras Nov 9, 2016
8308879
Fixed issue
sras Nov 9, 2016
74e8743
Refactor
sras Nov 9, 2016
a8c94fd
Make created fields optional and updated field required for writing
sras Nov 9, 2016
74eff26
Removed unused AllowAmbiguousTypes extension
sras Nov 9, 2016
290cc61
Reenable create functions
sras Nov 9, 2016
945ef75
Merge OpaleyeTypes with OpaleyeDef
sras Nov 10, 2016
e770e4c
Conver names to camel case and drop id from update functions
sras Nov 10, 2016
04fc93d
Make id fields read only and remove unused Default instances
sras Nov 10, 2016
93b6037
Fix warnings
sras Nov 10, 2016
3734c41
Stylize
sras Nov 10, 2016
07304e8
Refactor to use listToMaybe functions
sras Nov 10, 2016
1a298cb
Start of Audtable api implementation
sras Nov 12, 2016
dd2d971
Convert all api function to use AuditM
sras Nov 12, 2016
bd209c5
Add wrapper to opaleye functions that runs in AuditM
sras Nov 12, 2016
1004ca1
Add functions that does raw db updates in AuditM
sras Nov 14, 2016
4b32bdd
Rename AuditM to AppM
sras Nov 15, 2016
ea708d7
Added TH functions to create auditable lenses
sras Nov 15, 2016
6f80c26
Generate typeclasses for the auditable wrapper using TH
sras Nov 16, 2016
f940de5
Clean up warnings and start conversion of audit log to json Value type
sras Nov 16, 2016
18e2dfe
Change the audit log to use the json Value object
sras Nov 16, 2016
c6f7d14
Added type syns for AuditM wrapper
sras Nov 16, 2016
03a33cc
Convert all the api function to run in AppM
sras Nov 17, 2016
7b4ba29
Use Auditable models in api functions
sras Nov 17, 2016
6ebf760
Insert log to db on update calls
sras Nov 17, 2016
79d5114
Test case for audit logs
sras Nov 17, 2016
f02c93a
Fix build issues
sras Nov 18, 2016
a995b2d
Fix typo in TH function name
sras Nov 20, 2016
4a38268
Adding comments
sras Nov 21, 2016
14e2d8a
Ammende TH functions to take types that are not sysnonyms into account
sras Nov 25, 2016
4ce8966
Added schema files
sras Nov 29, 2016
4b3fca4
Added list tenants endpoints for load testing
sras Nov 30, 2016
82d2d78
Send mail via sendgrid
sras Dec 5, 2016
69f172f
Added email templates and linked mail sending from create tenant json…
sras Dec 7, 2016
aa38a20
Add inline attachment embedding using cid method
sras Dec 7, 2016
95f65e2
Fix double quoting of links and add hardcoded key
sras Dec 7, 2016
27cf3b3
Fix double quote issue with quasiquoter and add a hardcoded key to th…
sras Dec 7, 2016
480119f
Send mail from another thread and code refactor
sras Dec 7, 2016
741ff7d
Add user services
sras Dec 8, 2016
87b9674
Review modifications
sras Dec 9, 2016
81f6fab
Cleaned up stack.yaml
sras Dec 9, 2016
0a8c997
Use the updated mime-email library
sras Dec 12, 2016
29a8a80
Wrapped the app monad around ExceptT to enable handling of uncaught e…
sras Dec 12, 2016
65c5ccc
Trying to add request body to the error logs
sras Dec 13, 2016
52c102b
Use a helper function to fetch env info and log it in case of an exce…
sras Dec 13, 2016
0ce8cbf
Add request info to the log
sras Dec 13, 2016
cd991a1
Added airbrake source
sras Dec 14, 2016
e788e65
cabale file update
sras Dec 14, 2016
6ec820b
Wrap the auditable types
sras Dec 15, 2016
f56c9c5
Got the refactor to compile
sras Dec 15, 2016
caa39c0
Rename files
sras Dec 16, 2016
117f8db
Moved code to a separate package AppCore
sras Dec 16, 2016
54dcd2e
Added removeAuditableRow function to api base
sras Dec 16, 2016
39dbab3
Hide stuff in AppCore module
sras Dec 16, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions SpockOpaleye/SpockOpaleye.cabal
Original file line number Diff line number Diff line change
Expand Up @@ -16,23 +16,26 @@ cabal-version: >=1.10
library
hs-source-dirs: src
exposed-modules: Lib,
DataTypes,
ApiBase,
TenantApi,
UserApi,
RoleApi,
OpaleyeDef,
CryptoDef,
JsonInstances,
Validations,
DataTypes
Validations
build-depends: base >= 4.7 && < 5
,product-profunctors
,bytestring
,opaleye
,time
,postgresql-simple
,bcrypt
,text
,lens
,mtl
,vector
,Spock >=0.11
,aeson
default-language: Haskell2010
Expand All @@ -49,6 +52,7 @@ executable SpockOpaleye-exe
, lens
, text
, bcrypt
, vector
, aeson
default-language: Haskell2010

Expand Down
21 changes: 9 additions & 12 deletions SpockOpaleye/app/Main.hs
Original file line number Diff line number Diff line change
Expand Up @@ -34,15 +34,12 @@ app :: SpockM Connection MySession MyAppState ()
app = do
post ("tenants/new") $
do maybe_tenant_incoming <- jsonBody
maybe_newtenant <-
case maybe_tenant_incoming of
Just incoming_tenant -> do
result <-
runQuery (\conn -> validateIncomingTenant conn incoming_tenant)
case result of
Valid -> runQuery (\conn -> create_tenant conn incoming_tenant)
_ -> return Nothing
Nothing -> return Nothing
case maybe_newtenant of
Just tenant -> json tenant
_ -> json $ T.pack "Tenant not created"
case maybe_tenant_incoming of
Just incoming_tenant -> do
result <- runQuery (\conn -> validateIncomingTenant conn incoming_tenant)
case result of
Valid -> do
new_tenant <- runQuery (\conn -> create_tenant conn incoming_tenant)
json new_tenant
_ -> json $ T.pack "Validation fail"
Nothing -> json $ T.pack "Unrecognized input"
47 changes: 47 additions & 0 deletions SpockOpaleye/src/ApiBase.hs
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE FunctionalDependencies #-}
{-# LANGUAGE MultiParamTypeClasses #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE TemplateHaskell #-}

module ApiBase where

import Control.Lens
import qualified Data.Profunctor.Product.Default as D
import Data.Time (UTCTime, getCurrentTime)
import Database.PostgreSQL.Simple
import DataTypes
import Opaleye
import OpaleyeDef
import OpaleyeTypes
import Prelude hiding (id)

create_row ::(
HasCreatedat columnsW (Maybe (Column PGTimestamptz)),
HasUpdatedat columnsW (Column PGTimestamptz),
D.Default Constant incoming columnsW, D.Default QueryRunner returned row)
=> Connection -> Table columnsW returned -> incoming -> IO row
create_row conn table item = do
current_time <- fmap pgUTCTime getCurrentTime
let itemPg = (constant item) & createdat .~ (Just current_time) & updatedat .~ (current_time)
fmap head $ runInsertManyReturning conn table [itemPg] (\x -> x)

update_row :: (
HasUpdatedat haskells UTCTime
, D.Default Constant haskells columnsW
, D.Default Constant item_id (Column PGInt4)
, HasId haskells item_id
, HasId columnsR (Column PGInt4)
)
=> Connection -> Table columnsW columnsR -> item_id -> haskells -> IO haskells
update_row conn table it_id item = do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove it_id from the function signature. We're already assuming that we have a field called id in the item itself. What is the advantage of passing the id separately?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passing the id separately explicitly specify which record to update. The record contains the data that the row should be updated with. The id specify the row that should be updated. Since those are separate things, I think those must remain separate.

Imagine a case when someone have a record where they have made some updates to the fields, and want to save to a certain Id. But imagine that they missed to change the id field. When they make the call, the wrong record is going to get updated. There is nothing warning the user that they didn't specify the id of the row that should be updated, because that was implicitly assumed to be the id that came with the updated record.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Imagine a case when someone have a record where they have made some updates to the fields, and want to save to a certain Id.

Whoa... stop right there. You don't get to "pick" which record should be saved to which ID! The ID is part of the record and self-identifies its row.

But imagine that they missed to change the id field.

Which is the reason why ID should be a readonly field in the first place. I have yet to come across a valid use-case where one is mutating the primary key (ID) of a record -- either in the DB itself, or a record/object which represents the row in the DB.

Do you have a valid use-case for doing this?

Copy link
Contributor Author

@sras sras Nov 10, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you are using an ORM that provides an Identity map and in a language with mutable values, you can mutate the fields of the object and call the update method to sync itself with the database.

But in Haskell, owing to it's immutable nature, this pattern is not possible. So in haskell, there cannot be a unique instance that represent the data for an entity. For example, a User record cannot be made to uniquly represent a row in the user table. Because that would require that another user record that represent the same row does not exist at the same time.

In a ORM with an Identity map, if a = user.post and user2 = a.author, then user and user2 refer to the same object. But in haskell, user and user2 can be different records, even though they hold same values, including the ID.

This means, we cannot treat records as authoritative representations of the underlying domain objects, and hence cannot use patterns that depend on this assumptions IMHO.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But in Haskell, owing to it's immutable nature, this pattern is not possible. So in haskell, there cannot be a unique instance that represent the data for an entity. For example, a User record cannot be made to uniquly represent a row in the user table

What would you call the record-type that Opaleye returns, which has the ID inside of it? The ID is the database identity of the record, which is what we're bothered with.

In fact, if we want to be pedantic about it, I would NOT EXPORT the id setter lens at all. No one should be able to change the id of a record which represents a row in the DB. It has no practical purpose, and will result in bad things.

This means, we cannot treat records as authoritative representations of the underlying domain objects, and hence cannot use patterns that depend on this assumptions IMHO.

As soon as the DB moves out of the database and goes to any other "temporary holding area" (record in Haskell, data-structure in Redis, etc.) it is no longer an "authoritative representation" at all. So, I'm unsure of how this really elucidates the Haskell or non-Haskell way of doing things.

And what exactly are we trying to prevent here? Can you give a real-life example of the use-case you are envisioning?

Imagine a case when someone have a record where they have made some updates to the fields, and want to save to a certain Id.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ID is the database identity of the record, which is what we're bothered with.

A database row can only have one state at a given time. If you want a in-memory representation of a database row, you have to make sure that there are no more than one instance of it at the same time in the scope. If you cannot gaurantee that, then you will have inconstant state in your app, when there are conflicting representations of the same database row, and upon saving to the db, you will lose data.

In fact, if we want to be pedantic about it, I would NOT EXPORT the id setter lens at all. No one should be able to change the id of a record which represents a row in the DB. It has no practical purpose, and will result in bad things.

I am not sure preventing updates to id will make any difference. Because either way, you are not mutating in place. So there is really no difference between updating id, and updating all the fields while keeping the id same. Because they both are different instances.

There is no difference between the result of the following operations

Starting with two User records,

u1 = User {id = 10, name = "max", age= 20}
u2 = User {id = 15, name = "bob", age= 25}

Updating the id field of u1

u3 = u1 & id .~ 15

is equalent to updating the name and age of u2 to that of u1, while keeping the id same

u3 = u2 & name .~ "max" & age .~ 20

In both cases u3 has same values and is a different instance from either u3 or u2.

As soon as the DB moves out of the database and goes to any other "temporary holding area" (record in Haskell, data-structure in Redis, etc.) it is no longer an "authoritative representation" at all.

It is still authoritative representation even after it leaves the DB. In fact, once it leaves the DB, the row in the DB cease to be the authoritative representation, because the data might be getting mutated as it passes through the app. The row in the DB won't be valid until the entity gets saved back to the database.

And what exactly are we trying to prevent here? Can you give a real-life example of the use-case you are envisioning?

What we are trying to prevent is ending up with inconsistent data. I gave you an example earlier. You reply was

Whoa... stop right there. You don't get to "pick" which record should be saved to which ID! The ID is part of the record and self-identifies its row.

But that breaks down as soon as you have multiple instances of a record with the same ID.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But that breaks down as soon as you have multiple instances of a record with the same ID.

I'm sorry, but I'm not following your argument here. If you really want to get behind this argument, then the data-structures should be modelled on the lines of what Persistent has; wherein the identity (primary key) is not part of the value-record (other fields of the row). However, in Opaleye the ID is part of the record itself. So, there is no point mixing the two paradigms of thought over here.

Inconsistency is written all over this function signature:

updateTenant :: TenantId -> Tenant -> IO Tenant

There are two possible places where TenantId can be taken from. Which one is right? Why are there two places in the first place?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What we are trying to prevent is ending up with inconsistent data. I gave you an example earlier.

I meant an actual piece of code that you would genuinely write this way:

  • Fetch a record from the DB (and the record contains an ID field)
  • Change the values, but not the ID to obtain a new record (because of immutability and all that)
  • Write the record back to the DB, but to overwrite a different row

current_time <- getCurrentTime
let updated_item = (put_updated_timestamp current_time) item
runUpdate conn table (\_ -> constant updated_item) match_func
return updated_item
where
put_updated_timestamp :: (HasUpdatedat item (UTCTime)) => UTCTime -> item -> item
put_updated_timestamp timestamp = updatedat .~ timestamp
match_func :: (HasId cmR (Column PGInt4)) => (cmR -> Column PGBool)
match_func item = (item ^. id) .== (constant it_id)
88 changes: 57 additions & 31 deletions SpockOpaleye/src/DataTypes.hs
Original file line number Diff line number Diff line change
@@ -1,11 +1,22 @@
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE FlexibleContexts #-}
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE FunctionalDependencies #-}
{-# LANGUAGE MultiParamTypeClasses #-}
{-# LANGUAGE TemplateHaskell #-}
{-# LANGUAGE TypeSynonymInstances #-}

module DataTypes where

import Control.Lens
import CryptoDef
import Data.List.NonEmpty
import qualified Data.Profunctor.Product.Default as D
import Data.Text
import Data.Time (UTCTime)
import Data.Time (UTCTime, getCurrentTime)
import GHC.Generics
import Opaleye

data ValidationResult = Valid | Invalid
deriving (Eq, Show)
Expand All @@ -16,51 +27,66 @@ newtype TenantId = TenantId Int
data TenantStatus = TenantStatusActive | TenantStatusInActive | TenantStatusNew
deriving (Show, Generic)

data TenantPoly key name fname lname email phone status owner_id b_domain = Tenant
{ tenant_id :: key
, tenant_name :: name
, tenant_firstname :: fname
, tenant_lastname :: lname
, tenant_email :: email
, tenant_phone :: phone
, tenant_status :: status
, tenant_ownerid :: owner_id
, tenant_backofficedomain :: b_domain
} deriving (Show, Generic)
data TenantPoly key created_at updated_at name fname lname email phone status owner_id b_domain = Tenant {
_tenantpolyId :: key
, _tenantpolyCreatedat :: created_at
, _tenantpolyUpdatedat :: updated_at
, _tenantpolyName :: name
, _tenantpolyFirstname :: fname
, _tenantpolyLastname :: lname
, _tenantpolyEmail :: email
, _tenantpolyPhone :: phone
, _tenantpolyStatus :: status
, _tenantpolyOwnerid :: owner_id
, _tenantpolyBackofficedomain :: b_domain
} deriving (Show, Generic)

type Tenant = TenantPoly TenantId Text Text Text Text Text TenantStatus (Maybe UserId) Text

type TenantIncoming = TenantPoly () Text Text Text Text Text () (Maybe UserId) Text
type Tenant = TenantPoly TenantId UTCTime UTCTime Text Text Text Text Text TenantStatus (Maybe UserId) Text

type TenantIncoming = TenantPoly () () () Text Text Text Text Text () (Maybe UserId) Text
type TenantIncomingCreatable = TenantPoly () UTCTime UTCTime Text Text Text Text Text () (Maybe UserId) Text

data UserStatus = UserStatusActive | UserStatusInActive | UserStatusBlocked
deriving (Show)

newtype UserId = UserId Int
deriving (Show, Generic)

data UserPoly key tenant_id username password firstname lastname status = User {
user_id :: key,
user_tenantid :: tenant_id,
user_username :: username,
user_password :: password,
user_firstname :: firstname,
user_lastname :: lastname,
user_status :: status
data UserPoly key created_at updated_at tenant_id username password firstname lastname status = User {
_userpolyId :: key
, _userpolyCreatedat :: created_at
, _userpolyUpdatedat :: updated_at
, _userpolyTenantid :: tenant_id
, _userpolyUsername :: username
, _userpolyPassword :: password
, _userpolyFirstname :: firstname
, _userpolyLastname :: lastname
, _userpolyStatus :: status
}

type User = UserPoly UserId TenantId Text BcryptPassword (Maybe Text) (Maybe Text) UserStatus
type User = UserPoly UserId UTCTime UTCTime TenantId Text BcryptPassword (Maybe Text) (Maybe Text) UserStatus

type UserIncoming = UserPoly () () () TenantId Text Text (Maybe Text) (Maybe Text) ()

data Permission = Read | Create | Update | Delete
deriving (Show)

newtype RoleId = RoleId Int
deriving (Show)

data RolePoly key tenant_id name permission = Role
{ role_id :: key
, role_tenantid :: tenant_id
, role_name :: name
, role_permission :: permission
} deriving (Show)

type Role = RolePoly RoleId TenantId Text (NonEmpty Permission)
data RolePoly key tenant_id name permission created_at updated_at = Role {
_rolepolyId :: key
, _rolepolyTenantid :: tenant_id
, _rolepolyName :: name
, _rolepolyPermission :: permission
, _rolepolyCreatedat :: created_at
, _rolepolyUpdatedat :: updated_at
} deriving (Show)

type Role = RolePoly RoleId TenantId Text (NonEmpty Permission) UTCTime UTCTime
type RoleIncoming = RolePoly () TenantId Text (NonEmpty Permission) () ()

makeLensesWith abbreviatedFields ''RolePoly
makeLensesWith abbreviatedFields ''TenantPoly
makeLensesWith abbreviatedFields ''UserPoly
17 changes: 5 additions & 12 deletions SpockOpaleye/src/JsonInstances.hs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ module JsonInstances where
import Control.Monad
import Data.Aeson
import Data.Aeson.Types
import Data.Char
import Data.Text
import DataTypes

Expand All @@ -29,7 +30,7 @@ instance FromJSON TenantStatus where

instance FromJSON TenantIncoming where
parseJSON (Object v) =
(Tenant ()) <$> v .: "name" <*> v .: "firstname" <*> v .: "lastname" <*>
(Tenant () () ()) <$> v .: "name" <*> v .: "firstname" <*> v .: "lastname" <*>
v .: "email" <*>
v .: "phone" <*>
(pure ()) <*>
Expand All @@ -38,11 +39,7 @@ instance FromJSON TenantIncoming where

instance ToJSON TenantStatus where
toJSON = genericToJSON defaultOptions
toEncoding =
genericToEncoding
defaultOptions
{ constructorTagModifier = tg_modify
}
toEncoding = genericToEncoding defaultOptions { constructorTagModifier = tg_modify }
where
tg_modify :: String -> String
tg_modify "TenantStatusActive" = "active"
Expand All @@ -51,13 +48,9 @@ instance ToJSON TenantStatus where

instance ToJSON Tenant where
toJSON = genericToJSON defaultOptions
toEncoding =
genericToEncoding
defaultOptions
{ fieldLabelModifier = remove_prefix
}
toEncoding = genericToEncoding defaultOptions { fieldLabelModifier = (fmap Data.Char.toLower).remove_prefix }
where
remove_prefix = Prelude.drop 7
remove_prefix = Prelude.drop 11

instance ToJSON UserId where
toJSON = genericToJSON defaultOptions
Expand Down
Loading