Skip to content

Llapi updates #400

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Merged
merged 4 commits into from
Jun 9, 2020
Merged

Llapi updates #400

merged 4 commits into from
Jun 9, 2020

Conversation

DvirDukhan
Copy link
Collaborator

This PR allows:

  1. Retrieve model, script, and tensor Redis types from LLAPI so other modules can read/write AI keys
  2. Added LLAPI for variadic script input - modified the existing input and variadic input for integrity verification before adding new input. Added failure test for double variadic input.

@DvirDukhan DvirDukhan requested a review from lantiga June 4, 2020 20:49
@CLAassistant
Copy link

CLAassistant commented Jun 4, 2020

CLA assistant check
All committers have signed the CLA.

@codecov
Copy link

codecov bot commented Jun 4, 2020

Codecov Report

Merging #400 into master will decrease coverage by 0.75%.
The diff coverage is 44.82%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #400      +/-   ##
==========================================
- Coverage   73.22%   72.46%   -0.76%     
==========================================
  Files          21       21              
  Lines        4359     4384      +25     
==========================================
- Hits         3192     3177      -15     
- Misses       1167     1207      +40     
Impacted Files Coverage Δ
src/model.c 69.20% <0.00%> (-0.40%) ⬇️
src/redisai.h 0.00% <0.00%> (ø)
src/tensor.c 83.50% <ø> (-0.29%) ⬇️
src/script.c 63.78% <47.36%> (-15.87%) ⬇️
src/redisai.c 77.09% <100.00%> (+0.15%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6e65a42...f182739. Read the comment docs.

@lantiga
Copy link
Contributor

lantiga commented Jun 5, 2020

@filipecosta90 I'm ok to merge, but in perspective (sooner than later) we should have a C++ test suite

@gkorland gkorland requested review from lantiga and filipecosta90 June 9, 2020 06:28
@lantiga lantiga merged commit fcdf0d2 into master Jun 9, 2020
@lantiga lantiga deleted the llapi_updates branch June 9, 2020 09:20
filipecosta90 pushed a commit that referenced this pull request Aug 24, 2020
* added low level api return redis types

* added variadic to llapi

* fixed memory issue for params array re-alloc
@filipecosta90 filipecosta90 mentioned this pull request Aug 24, 2020
filipecosta90 added a commit that referenced this pull request Sep 2, 2020
* Introduced readies submodule (#377)

* Introduced readies submodule

* Fix in paella

* Updated get_deps.sh and docs

* [WIP] add C API reference to docs using auto tool  (#368)

* [add] add C API reference to docs using auto tool ( python3 generate_llapi_reference.py )

* [add] generating C API reference on the fly

* [add] moving api_reference under Developer Nav

Co-authored-by: Luca Antiga <luca.antiga@orobix.com>

* [WIP] Enable AI.SCRIPTRUN on AI.DAGRUN* (#383)

* [add] decoupled scriptrun command parsing from runtime scriptrun.

* [add] split positive/negative tests on pytorch scriptrun.

* [add] refactor AI.DAGRUN_RO and AI.DAGRUN to use the same code base (with read/write modes)

* [add] added positive and negative tests for dagrun with scriptrun

* [add] updated documentation to reflect scriptrun support on dagrun

* [add] added example enqueuing multiple SCRIPTRUN and MODELRUN commands within a DAG

* Add support for variadic arguments to SCRIPT (#395)

* Add support for variadic arguments to SCRIPT

* Add negative errors

* atomic ref count (#403)

* Multi-platform build (#398)

* CircleCI: increased GPU test timeout to 40m (#404)

* CI: platform-build-defs -> on-master-and-version-tags (#405)

* Llapi updates (#400)

* added low level api return redis types

* added variadic to llapi

* fixed memory issue for params array re-alloc

* Avoid splitting outputs in batches when nbatches == 1 (#406)

* Avoid splitting outputs in batches when nbatches == 1

* Add batch size checks

* Fix batch checks

* Update readies

* Add bad batching test

* Update README.md

* CircleCI: fixed redis installation in coverage builds (#423)

* Docker6.0.5 (#427)

* Update Dockerfile

* Update Dockerfile.arm

* Update Dockerfile.gpu

* submodule pull moved up (#428)

* Update mkdocs.yml (#432)

* [add] Add relevant RedisAI config entries to the INFO output, so that standard monitoring systems would be able to monitor them. (#396)

* Updated support email and Orobix to Tensorwork on ramp file (#436)

* [add] updated support email and Orobix to Tensorwork on ramp file

Co-authored-by: Sherin Thomas <sherin@tensorwerk.com>

* CircleCI: Fixed problem with GPU testing (#440)

* Fixed platforms build problem (#441)

* Fixed platforms build problem

* Build: updated Redis versions

* Build: another Redis version update

* Dependencies in ramp.yml (#444)

* Fixed flagged as "getkeys-api" during the registration ( AI.DAGRUN, AI.DAGRUN_RO, AI.MODELRUN, AI.SCRIPTRUN ) (#438)

* [wip] wip on fixing the commands being flagged as getkeys-api during the registration. ( AI.MODELRUN, AI.SCRIPTRUN, AI.DAGRUN, AI.DAGRUN_RO )

* [add] pytorch oss cluster tests passing

* [add] TensorFlow lite tests passing on oss cluster

* [add] ONNX tests passing on oss cluster

* [add] TensorFlow tests passing on oss cluster

* [wip] wip on dag

* [add] DAG tests passing on oss cluster

* [add] enabling oss cluster tests on CI

* [add] bumping rmbuilder version from 6.0.1 to 6.0.5 on circleci

* [fix] fixed potential invalid mem accesses on RedisAI_DagRunSyntaxParser, RAI_parseDAGPersistArgs

* [fix] fixed RAI_FreeDagOp wrongfully calling RedisModule_Free on dagOp->inkeys, dagOp->outkeys

* [fix] fixed dictKey allocation on RAI_parseDAGLoadArgs

* [add] alter Sanitizer tests to accommodate keys on the same slot.

* [add] extracted the methods to respond to RedisModule_IsKeysPositionRequest() from main code of dagrun,modelrun, and scriptrun to specific methods

* [fix] Fixes per PR review

* [fix] fixes per PR review

* [fix] fix per CI ascii conversion error on test_dagro_modelrun_financialNet_no_writes_multiple_modelruns

* Safely add to arrays + fix for #443 (#449)

* Make sure we reassign the pointer in array_append

* Fix case-sensitive comparison for devicestr

* Fix sanitizer tests

(cherry picked from commit 5c7813e)

* fixed tests messages

* Shallow copy persisted tensor

* Resolve log format warnings

* Fix error message and test

* Fix memory management in local context dict

* Fixed artifacts handling + added tests logs aggregation (#445)

* Snapshot packages are placed into http://redismodules.s3.amazonaws.com/redisai/snapshots. Private branches (i.e., non-master) are also supported, just need to enable deploy-snapshot in config.yml to fire on on-any-branch rather than just on-master.
* Release packages are placed into http://redismodules.s3.amazonaws.com/redisai, and it's going to be crowded in there, so we may want to consider putting each version into its own directory.
* `BB` in-source breakpoints are now supported (in `DEBUG=1` builds): put `BB;` inside the code and it will stop if you're running under gdb.
* Tests results in CircleCI are now aggregated as artifacts.

* Build: setuptools-related fix (#455)

(cherry picked from commit d1fcd0d)

Co-authored-by: Rafi Einstein <raffapen@outlook.com>
Co-authored-by: Luca Antiga <luca.antiga@orobix.com>
Co-authored-by: DvirDukhan <dvir@redislabs.com>
Co-authored-by: Guy Korland <gkorland@gmail.com>
Co-authored-by: Sherin Thomas <sherin@tensorwerk.com>
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants