noth/surrealpatch
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1110 commits 2 branches 0 tags 27 MiB
Commit graph

298 commits

Author SHA1 Message Date
Tobie Morgan Hitchcock
0615896c0e Add timeout errors to json output 2018-05-09 22:52:47 +01:00
Tobie Morgan Hitchcock
5442861cb8 Enable global ENV variable to detect environment 2018-05-09 04:08:04 +01:00
Tobie Morgan Hitchcock
ebd5680b7d Don’t display live queries in SQL INFO statements 2018-05-08 18:53:21 +01:00
Tobie Morgan Hitchcock
e3bea0a897 Use context.Context on all kvs requests 2018-05-02 03:05:03 +01:00
Tobie Morgan Hitchcock
b7d89ee65a Add support for foreign tables with group by clauses 2018-05-02 02:43:04 +01:00
Tobie Morgan Hitchcock
7f6d1565ff Enable ability to set a global query timeout duration 2018-05-02 02:42:16 +01:00
Tobie Morgan Hitchcock
9f0ea3ee61 Improve database query logging 2018-05-02 02:32:06 +01:00
Tobie Morgan Hitchcock
47e1a4aceb Fix a bug with virtual records in scoped queries 
When querying a virtual document, it would check that the scope had permission to view the record, even though the record did not actually belong to a table.

Not it checks to see if the record is a virtual in-memory record, and does not perform any permissions checks if this is the case.
 2018-05-01 13:17:10 +01:00
Tobie Morgan Hitchcock
f5a0739985 Add foreign table information to SQL INFO query 2018-04-28 22:08:34 +01:00
Tobie Morgan Hitchcock
ec8ece9878 Remove for loop from mutex 2018-04-28 22:08:19 +01:00
Tobie Morgan Hitchcock
f335d71aba Move to channel based mutex 2018-04-28 20:35:20 +01:00
Tobie Morgan Hitchcock
99d050b238 Ensure linked records are fetched using the same query version 2018-04-28 20:35:19 +01:00
Tobie Morgan Hitchcock
14c0d93635 Ensure arrays and objects are always set 2018-04-27 15:23:58 +01:00
Tobie Morgan Hitchcock
ec6a44f2d6 Regenerate codec files 2018-04-27 00:46:43 +01:00
Tobie Morgan Hitchcock
444a38986a Ensure field permissions are correctly applied 2018-04-27 00:40:59 +01:00
Tobie Morgan Hitchcock
eb62515a05 Ensure that fields are not removed when set to NULL 2018-04-27 00:40:36 +01:00
Tobie Morgan Hitchcock
9b03178dfd Improve live query performance and simplicity 2018-04-27 00:04:36 +01:00
Tobie Morgan Hitchcock
5d5fdc296f Include nested fields in SQL INFO query output 2018-04-25 00:55:49 +01:00
Tobie Morgan Hitchcock
a62797e7da Prevent race conditions in live query socket notifications 2018-04-25 00:01:29 +01:00
Tobie Morgan Hitchcock
672d298e7e Make db package more efficient 2018-04-25 00:00:36 +01:00
Tobie Morgan Hitchcock
ae303d062c Add FETCH expressions to LIVE SELECT queries 2018-04-24 23:59:49 +01:00
Tobie Morgan Hitchcock
47ef2de60f Prevent concurrent record edits and race conditions 2018-04-24 17:11:12 +01:00
Tobie Morgan Hitchcock
db9534a2d6 Ensure empty arrays / objects are equal to EMPTY 2018-04-24 15:58:45 +01:00
Tobie Morgan Hitchcock
fa1061b3a5 Use query version time from executor not document 2018-04-24 15:57:41 +01:00
Tobie Morgan Hitchcock
add47a5ada Ensure correct ordering with same-compare values 2018-04-24 15:54:50 +01:00
Tobie Morgan Hitchcock
11a9fa05e6 Prevent race conditions in when fetching document fields 2018-04-24 13:53:15 +01:00
Tobie Morgan Hitchcock
27a5f785fd Change ERR_KV to ERR_EX for ‘record already exists’ errors 2018-04-23 09:12:00 +01:00
Tobie Morgan Hitchcock
1f30035899 Add SQL FETCH functionality to SELECT statements 2018-04-22 00:10:52 +01:00
Tobie Morgan Hitchcock
35047ce04c Improve remote array record fetching 2018-04-22 00:10:51 +01:00
Tobie Morgan Hitchcock
fb256df42b Prevent infinite loops with nested subqueries 2018-04-20 23:40:52 +01:00
Tobie Morgan Hitchcock
d2a451345a Don’t alter subquery RETURN type 
Don’t alter the RETURN type for CREATE / UPDATE / DELETE / RELATE / INSERT / UPSERT statements. Not only does this cause unexpected behaviour, but it also altered the statement from multiple goroutines, and therefore caused a data race.
 2018-04-20 23:35:32 +01:00
Tobie Morgan Hitchcock
459c7264ec Don’t enable array lengths fetch using data paths 2018-04-20 21:33:13 +01:00
Tobie Morgan Hitchcock
8d11027788 Fix bug in downstream cork package 
A bug in the cork encoding package prevented encoding of large maps or arrays.
 2018-04-20 16:40:40 +01:00
Tobie Morgan Hitchcock
83edea8a66 Don’t fetch right side of AND / OR statements if not necessary 2018-04-20 00:17:13 +01:00
Tobie Morgan Hitchcock
c4f8448345 Ensure session wide variables are available on a live query context 2018-04-19 13:36:08 +01:00
Tobie Morgan Hitchcock
4d155c61ce Only run events when the document has changed 2018-04-19 13:36:04 +01:00
Tobie Morgan Hitchcock
e8accd1b8e Enable limiting the maximum number of query workers 2018-04-18 15:49:32 +01:00
Tobie Morgan Hitchcock
07d52831d0 Improve tests for DEFINE EVENT db queries 2018-04-18 15:27:51 +01:00
Tobie Morgan Hitchcock
cf68ebffb1 Improve DB error message types 2018-04-18 15:27:21 +01:00
Tobie Morgan Hitchcock
6fe4db6a58 Check permissions on the initial document when deleting 
When deleting a document, we need to check the visibility permissions using the initial document, as the current document will now have been emptied. As a result, no delete notifications were being delivered to LIVE queries.
 2018-04-15 01:53:36 +01:00
Tobie Morgan Hitchcock
49ec33987f Update generated code 2018-04-14 21:59:16 +01:00
Tobie Morgan Hitchcock
7602b77c6d Store regex values in special sql.REGEX type 2018-04-14 21:56:29 +01:00
Tobie Morgan Hitchcock
dc700c34f6 Update 3rd party package dependencies 2018-04-14 19:24:25 +01:00
Tobie Morgan Hitchcock
bc31140308 Improve live query notification delivery 2018-04-14 19:20:15 +01:00
Tobie Morgan Hitchcock
677767d85b Ensure sockets are deregistered on disconnect 2018-04-14 19:17:07 +01:00
Tobie Morgan Hitchcock
67cfca04b9 Enable permissions on individual document fields 2018-04-14 19:14:47 +01:00
Tobie Morgan Hitchcock
2a74759a71 Enable IF and RUN statements in MultStatements 2018-04-14 18:36:28 +01:00
Tobie Morgan Hitchcock
6354d0ce2a Fix comment typos 2018-04-14 18:02:58 +01:00
Tobie Morgan Hitchcock
ac19b552e5 Add SQL RUN query statement type 2018-04-14 17:55:05 +01:00
Tobie Morgan Hitchcock
ea1749f03f Split DB event processing code into separate file 2018-04-13 21:15:38 +01:00
Tobie Morgan Hitchcock
3f7d7fc863 Add fuzzy string search to conditional SQL clauses 2018-04-13 20:34:52 +01:00
Tobie Morgan Hitchcock
9b4ff941b6 Ensure events are run with DB level permissions 2018-04-10 07:19:22 +01:00
Tobie Morgan Hitchcock
93b11514c5 Move db.yield code into separate file 2018-04-10 07:18:25 +01:00
Tobie Morgan Hitchcock
be832029a7 Remove google and syslog logging 2018-04-05 08:51:44 +01:00
Tobie Morgan Hitchcock
0dc9ad339c Enable multiple expressions with database events 
It is now possible to run multiple query expressions when an event on a table has occured. Query expressions can be separated with a semicolon, and will be run in the same transaction as the main query.
 2018-04-04 19:20:07 +01:00
Tobie Morgan Hitchcock
86a911b4a8 Improve tests for DB indexes 2018-04-04 19:16:32 +01:00
Tobie Morgan Hitchcock
7c39c99bf3 Ensure correct index values are inserted 2018-04-01 00:33:51 +01:00
Tobie Morgan Hitchcock
160986cc0b Shorten test timeout time to prevent test failure 2018-03-19 11:35:31 +00:00
Tobie Morgan Hitchcock
b4a8f10cdc Ensure query variables are consistent 
All request variables which are assigned to an sql query are now specified in one location, ensuring that they are consistent and always present for all queries.
 2018-03-18 21:38:21 +00:00
Tobie Morgan Hitchcock
ad4adbd986 Enable select * data from a foreign thing / record 2018-03-18 21:33:26 +00:00
Tobie Morgan Hitchcock
ca392f87a7 Add ON SIGNUP and ON SIGNIN events to SQL SCOPE 
It is now possible to run queries when a user signs-up or signs-in, by specifying multiple queries within an ON SIGNUP (…) clause, or a ON SIGNIN (...) clause.
 2018-03-18 21:30:02 +00:00
Tobie Morgan Hitchcock
069baf71b7 Empty strings now match an EMPTY sql clause 2018-02-17 09:23:51 +00:00
Tobie Morgan Hitchcock
87cf68b213 Don’t log context auth data 2018-02-14 15:48:55 +00:00
Tobie Morgan Hitchcock
b4dfaf2898 Don’t assert the value of a field type 
Previously if a field had a type, and a value was entered which did not match that type, then an error was raised, even though the ASSERT clause was not set. Now the field will be set to nil if the field does not match the specified type, and the ASSERT clause can be used to ensure that a valid value is always entered.
 2018-02-14 15:48:42 +00:00
Tobie Morgan Hitchcock
31c7849643 No need to cast, incase the value is not set on the context 2018-02-14 13:15:56 +00:00
Tobie Morgan Hitchcock
e073bf2775 Enable persistent request variables 
It is now possible to set a variable on a connection, and use that variable in multiple subsequent queries. This ensures that websockets can make use of state variables for detecting logged-in, and non-logged-in clients.
 2018-02-12 22:13:36 +00:00
Tobie Morgan Hitchcock
9b0ef8d6d7 When clearing an index, remove all items 2018-02-12 09:05:52 +00:00
Tobie Morgan Hitchcock
02ef06a1d2 Increase the limit when fetching batch items from the kvs 2018-02-11 00:17:16 +00:00
Tobie Morgan Hitchcock
ccf57df085 Add support for mysql as a backend kv store 2018-02-11 00:16:28 +00:00
Tobie Morgan Hitchcock
84b72e25d2 Don’t use of ClrL, GetL, DelL, PutL methods on KV store 
Remove the need to query a particular KV level, and instead use slightly different encoded keys, and use prefix based iteration. This means we can use different KV stores to give the same functionality, without needing to use hierarchical KV stores.
 2018-02-09 15:08:27 +00:00
Tobie Morgan Hitchcock
8cce4ad185 No need for PutC as we always already have the record value 
We don’t need to check whether the record already exists, as we always have any record contents when we are procesing a document. Therefore we already know if a document exists or not in the kv store.
 2018-02-08 10:59:54 +00:00
Tobie Morgan Hitchcock
dc6a357e26 Enable passing context when beginning a transaction 2018-02-06 17:07:42 +00:00
Tobie Morgan Hitchcock
36e7d8ed3a Flush websocket notifications correctly 
Websocket notifications were cleared/flushed regardless of whether individual statements were successful or not.

Now notifications are shifted onto the stack, or removed if the statement is unsuccessful. Once the full query has been processed, all pending notifications are flushed to all websockets (ignoring the current connection frin which th query originated).
 2018-01-31 09:15:29 +00:00
Tobie Morgan Hitchcock
7a3355bff0 Enable customisable file size policies for storage backends 2018-01-12 11:10:11 +00:00
Tobie Morgan Hitchcock
d0d1316449 Add test for parsing times and records within json 2018-01-10 14:54:09 +00:00
Tobie Morgan Hitchcock
2f23e84877 Parse times and records within json content 2018-01-10 13:17:50 +00:00
Tobie Morgan Hitchcock
19bf0b3e7d Don’t ignore errors from the data layer 
If the data layer encountered an error when committing a transaction, then the error was ignored. Now all errors from the data layer which occur when cancelling or committing a transaction are passed to the end-user and displayed accordingly.
 2018-01-10 11:33:26 +00:00
Tobie Morgan Hitchcock
6c939c756c Fix incremental table processing in iterator 2018-01-10 10:36:30 +00:00
Tobie Morgan Hitchcock
e24cc5b620 Add method variable to events 
A new $method variable is now available in database events. The $method variable specifies the type of request that was made which triggered the event. Possible values are CREATE / UPDATE / DELETE.
 2017-12-20 06:53:06 +00:00
Tobie Morgan Hitchcock
4e81a5633c Improve binary condition comparison checks 2017-12-12 01:11:09 +00:00
Tobie Morgan Hitchcock
b4c49a7099 Ensure that the NS and DB are available to the live query 2017-12-12 01:08:32 +00:00
Tobie Morgan Hitchcock
21c7cb2341 Remove unnecessary brackets in sql test expressions 2017-12-12 01:07:24 +00:00
Tobie Morgan Hitchcock
91bfb043dc Add test for events which create foreign keys 2017-12-08 14:30:11 +00:00
Tobie Morgan Hitchcock
fd35a134ca Enable special param values which supersede subquery params 2017-12-08 14:28:55 +00:00
Tobie Morgan Hitchcock
5357c93558 Remove @ character for record ids in tests 2017-12-08 14:27:47 +00:00
Tobie Morgan Hitchcock
e3c7aacd66 Enable parameters in LIVE queries 2017-12-08 10:29:52 +00:00
Tobie Morgan Hitchcock
84b9cdc93a Fix binary equality tree parsing and calculations. 2017-12-06 20:18:57 +00:00
Tobie Morgan Hitchcock
fc0825a7fa Make sure that any non floats are converted to floats 2017-12-06 13:21:49 +00:00
Tobie Morgan Hitchcock
548995e935 Cancel the context timeout AFTER errors are checked 2017-12-06 13:21:12 +00:00
Tobie Morgan Hitchcock
6d8686138a Ensure permission checks don’t create subsequent checks 2017-12-06 13:20:34 +00:00
Tobie Morgan Hitchcock
7e8d5f9fe9 Add tests for using params with CONTENT/MERGE/DIFF 2017-12-05 11:09:06 +00:00
Tobie Morgan Hitchcock
6e8fdaa2db Ensure nil fields are removed when SCHEMAFULL 2017-12-05 11:09:04 +00:00
Tobie Morgan Hitchcock
ccdc6fb8ea Add ability to define the order fields are processed 2017-12-05 01:57:25 +00:00
Tobie Morgan Hitchcock
2087ea5fb5 Ensure ranges are iterated over correctly 2017-12-05 01:12:12 +00:00
Tobie Morgan Hitchcock
5c7378c9c7 Prevent circular references when yielding data 2017-12-04 22:20:39 +00:00
Tobie Morgan Hitchcock
8fadbc9f35 Enable path expressions starting from params 2017-12-04 22:20:19 +00:00
Tobie Morgan Hitchcock
444d7858b8 Enable use of VOID in field VALUE statements 
It is now possible to specify VOID values in field VALUE statements, to remove the field entirely.
 2017-12-04 10:10:29 +00:00
Tobie Morgan Hitchcock
7733b7b129 Reset ctx variables between calculating field VALUE and ASSERT 
If the field value was changed in the VALUE statement, then the context variables were not reset to reflect this change in the ASSERT statement. As a result the ASSERT would receive the old values prior to being affected by the VALUE statement.
 2017-12-04 10:09:49 +00:00
Tobie Morgan Hitchcock
d367726709 Ensure NULL/VOID/EMPTY values are compared correctly 2017-12-03 11:10:38 +00:00
Tobie Morgan Hitchcock
327bdd05cd Fix bug where CONTENT/MERGE/DIFF params weren’t fetched 2017-12-03 00:46:46 +00:00
Tobie Morgan Hitchcock
f623ded1e4 Simplify table records loop in iterator 2017-12-01 00:36:14 +00:00
Tobie Morgan Hitchcock
34aa10f9b8 Pass channels into goroutines to prevent data race 2017-12-01 00:35:33 +00:00
Tobie Morgan Hitchcock
4ff14caace Add initial path following implementation 2017-11-28 01:20:30 +00:00
Tobie Morgan Hitchcock
c925c40846 Yes we do need to copy all maps and slices 2017-11-27 18:32:47 +00:00
Tobie Morgan Hitchcock
4437b2b8a5 Remove old and unused code 2017-11-27 14:21:56 +00:00
Tobie Morgan Hitchcock
7fbc9feff9 Simplify uuid util package 2017-11-27 11:34:59 +00:00
Tobie Morgan Hitchcock
386efc73ff Add extra tests for parsing record IDs 2017-11-26 17:45:17 +00:00
Tobie Morgan Hitchcock
0762ee25b2 Use XIDs instead of UUIDs for record IDs 2017-11-26 16:54:49 +00:00
Tobie Morgan Hitchcock
1ac78aa950 A single *Thing in a subquery is the same as LIMIT 1 2017-11-26 13:45:48 +00:00
Tobie Morgan Hitchcock
851c52d901 No need to version index data 2017-11-24 12:46:12 +00:00
Tobie Morgan Hitchcock
37e015b482 Only return an executor to the pool once 
The executor was being returned to the pool twice, which was resulting in nil interface{} errors when creating new executors concurrently, as the transaction would not be set properly.
 2017-11-24 01:45:56 +00:00
Tobie Morgan Hitchcock
a8137134e3 Ensure erased document is nil not empty map 2017-11-17 12:41:35 +00:00
Tobie Morgan Hitchcock
43d841dacc Full update of the DB package 2017-11-16 20:53:39 +00:00
Tobie Morgan Hitchcock
fa714c1bdd Listen and wait for errors when server exits 2017-11-16 20:03:38 +00:00
Tobie Morgan Hitchcock
5a558c0d6f Implement Authable statements for improved logging 2017-03-24 21:25:22 +00:00
Tobie Morgan Hitchcock
7a071af821 Download generate dependencies in each gen files 2017-03-04 10:55:58 +00:00
Tobie Morgan Hitchcock
a133eb2482 Add HTTP 504 request timeouts to certain routes 2017-03-03 21:41:06 +00:00
Tobie Morgan Hitchcock
a96a8466cb Add initial golang code tracing and logging 2017-03-03 21:39:37 +00:00
Tobie Morgan Hitchcock
98db89a2d7 Implement query statement timeout functionality 2017-03-02 14:38:56 +00:00
Tobie Morgan Hitchcock
2d5151c8ba Enable SQL AST string formatting 2017-03-02 10:47:10 +00:00
Tobie Morgan Hitchcock
1157835a29 Don’t perform cpu intensive operations in SQL parser 2017-03-02 10:32:20 +00:00
Tobie Morgan Hitchcock
e3634fb4a8 Improve SQL debug logging 2017-02-28 00:19:21 +00:00
Tobie Morgan Hitchcock
b68c33835e Ensure all db.executor methods are private 2017-02-28 00:17:10 +00:00
Tobie Morgan Hitchcock
3167be73bb No need to pass statements in from goroutine 
A series of statements on one request are always handled and run serially, so there is no need to loop over them in a goroutine.
 2017-02-28 00:00:39 +00:00
Tobie Morgan Hitchcock
d5ee46ab17 Use chan struct for http connection closed channel 2017-02-27 15:30:46 +00:00
Tobie Morgan Hitchcock
7b78ff5b6e Add request context data to db query logs 2017-02-23 10:14:10 +00:00
Tobie Morgan Hitchcock
07c034bb75 Add request context to query executor 2017-02-23 10:14:02 +00:00
Tobie Morgan Hitchcock
73c56e4df5 Log sql queries centrally, not separately 2017-02-23 10:11:52 +00:00
Tobie Morgan Hitchcock
b304c9f3f4 Add RETURN NULL / VOID / EMPTY query functionality 2017-02-20 01:45:09 +00:00
Tobie Morgan Hitchcock
fbb0bd79bd Don’t delete txn until we have cancelled/committed 2017-02-20 01:44:13 +00:00
Tobie Morgan Hitchcock
a700ef9bc4 Improve debug logging throughout codebase 2017-02-20 01:44:13 +00:00
Tobie Morgan Hitchcock
6a719d815b Improve errors throught codebase 2017-02-20 01:44:13 +00:00
Tobie Morgan Hitchcock
e86857bdf7 Enable import / export of database transaction log 2017-02-20 01:44:12 +00:00
Tobie Morgan Hitchcock
363cfa8758 Use sync pool for query executor 2017-02-20 01:44:12 +00:00
Tobie Morgan Hitchcock
83a64c6c29 Enforce default RETURN value in sql parser 2017-02-20 01:44:12 +00:00
Tobie Morgan Hitchcock
623d55225c Use transaction mem cache for db config data 2017-02-20 01:44:12 +00:00
Tobie Morgan Hitchcock
1cb44cce2f Only product stack trace if log level is debug 2017-02-20 01:44:11 +00:00
Tobie Morgan Hitchcock
2a35485089 Output full stacktrace for errors when debugging 2017-02-20 01:44:11 +00:00
Tobie Morgan Hitchcock
7f0ea653b8 Improve sql query requests timing 2017-02-20 01:44:11 +00:00
Tobie Morgan Hitchcock
1fd450890b Fix typo 2017-02-20 01:44:10 +00:00
Tobie Morgan Hitchcock
a5dc358e4e Don’t pass txn around, but use executor txn 2017-02-20 01:44:08 +00:00
Tobie Morgan Hitchcock
6807d4c338 Switch underlying KV store to rixxdb/dendrodb 2017-02-20 01:31:15 +00:00
Tobie Morgan Hitchcock
a3b9362adb Switch underlying KV store to rixxdb/dendrodb 2017-02-09 11:16:59 +00:00
Tobie Morgan Hitchcock
e3c472b31a Remove incorrect build tags 2017-01-11 11:22:17 +00:00
Tobie Morgan Hitchcock
fa40f52302 Place build instruction before package name 2016-12-08 22:04:22 +00:00
Tobie Morgan Hitchcock
2fcc2e3668 Improve SQL INFO command to support namespaces 2016-11-23 12:53:27 +00:00
Tobie Morgan Hitchcock
94c9631d91 Improve database authentication 
Improve the database authentication implementation for namespaces, databases, and scopes.
 2016-11-21 18:48:25 +00:00
Tobie Morgan Hitchcock
4931bfbca5 Enable public method for running sql query 2016-11-21 14:36:37 +00:00
Tobie Morgan Hitchcock
85fcdccab5 Ensure maps are initialised before parsing sql 2016-11-05 13:59:39 +00:00
Tobie Morgan Hitchcock
54ad8a16d9 Add codec code generation 2016-11-04 11:45:38 +00:00