Create SmartFilters from file due to ARG_MAX

+2 votes

So,

I need to create a Transaction Filter which is enormous (angel) due to a few JSON's. There's simply no way to insert that on the terminal due to "getconf ARG_MAX" (the max size of a terminal command) generally being ~250k bytes.

Already tried to put everything on the same line without spaces, but I'm way pass that.

Can you help me in any way? 

asked May 22 by tloriato

2 Answers

+1 vote
Hi,

Try using a JsonRPC wrapper (for example Savoir). That way you can pass your smart filters code as function parameters.

You can find more on Savoir in this link: https://pypi.org/project/Savoir/
answered May 23 by mirabdulHaseeb
Great suggestion mirabdull, it didn't work, tho.
Using the NodeJS wrapper I got the error:   
{ code: -8,
 message: 'Invalid custom fields or filter name, too long' }  

And the same error code, albeit with a different message, using Savior:
Error! Status code: 500
Text: {"result":null,"error":{"code":-8,"message":"Couldn't compile filter code: SyntaxError: Unexpected identifier"},"id":1}  

I'm having to escape a lot of characters in Javascript, but I removed them to paste it on the python file. I'm trying to use """function ..... """ on Python, but unsure if I should escape something for it.
Please follow this syntax for Savoir:
api.create('txfilter', 'filtername', {} ,'function filtertransaction() { var tx=getfiltertransaction(); if (tx.create) return "Stream creation temporarily disabled"; }')

For more on Smart Filters visit: https://www.multichain.com/developers/smart-filters/ (Under Smart Filter Tutorial Section)
I'm calling it like that, but as my function spawns across multiple lines and has single quotes I tried to use """string""" in order to not need to escape all the single quotes.

I'm not a heavy python user tho, so if you think there's a significant difference between """x""" and 'x' that might be creating this situation let me know and I'll find a way to work with it!
"""string""" should work fine. For example:

stcode = """function filtertransaction() { var tx=getfiltertransaction(); if (tx.create) return "Stream creation temporarily disabled"; }"""

api.create('txfilter', 'filtername1', {} ,stcode)

PS: I'm sorry if I don't understand your question correctly.
Basically: either there's a hidden limit for the js-code that we don't know or I'm doing something wrong.

When using """ in Python, do I need to escape any character? For example, in Javascript I might have to do: 'driver'\'s license'
No, you don't have to escape anything. Try running a snippet of code to make sure there's no hidden limit.
Code in Python (3, to use with Python2 just remove .Savior from the first line) without the "ceps" and "municipios" objects (+1mi lines):
https://gist.github.com/tloriato/8aed7a8385ac7d0d1a6e75d48bfeb4e8

I'm still getting:

Error! Status code: 500
Text: {"result":null,"error":{"code":-8,"message":"Couldn't compile filter code: SyntaxError: Unexpected identifier"},"id":1}

I don't have a clue about why is that, it should have worked I think
Are you not using python3?

Please make sure your code's syntax is correct. it seems like you have an extra "}" in your code.
I'm using Python 3, yes. Where do you think there's an extra "} "? I just parsed it and it appears correct.
0 votes
You could try using the MultiChain Web Demo, which provides a web interface for many APIs, including compiling, testing, creating and approving transaction filters.
answered May 23 by MultiChain
The likely issue is that you are coming up against one of the size limits for transactions or in-transaction data (since a filter's code is placed inside a transaction output). See the max-std-op-return-size and max-std-tx-size blockchain parameters. You can start a new chain with higher limits, or if that's not an option, use the upgrade mechanism to increase them on the already running chain.
So, I realized that I could read the JS directly from the file with Python and use Savior to call the RPC with it, but I'm still getting the same error -8,  'Invalid custom fields or filter name, too long'. However, the file size now is pretty modest ~0.6mb.  


api = Savoir(rpcuser, rpcpasswd, rpchost, rpcport, chainname)

with open("deployReady.js", 'r') as js_file:
    js = js_file.read()
    print(api.create('txfilter', 'smartFilter1', {}, js))


The deployReady is here: https://gist.githubusercontent.com/tloriato/d3aa5df63c12cfec05c9b55f39f7fcfc/raw/51e16c97bc57aa88eb8a77aa2ffc17df1c85dbd1/file.js

Honestly, I'm clueless at this point
I have asked the developer team and will get back to you.
Thank you, I'll be waiting!
So... what I wrote above was incorrect and based on a wrong assumption - my apologies. In the current protocol there's a fixed limit of just under 64 kb of code. I can see how that's a problem in your case, and we're looking into raising this a new protocol version that will be available in the next maintenance release of MultiChain. Apologies in the meantime.
Thank you for the support! If it turns out to be a big trouble and you can't make it, feel free to point me in the direction and I'll try to do it myself!
...