Sporadically receiving connection error when doing JSON RPC requests

+1 vote
Hello,

I made a web application which sends JSON RPC requests to a multichain node on the same server (localhost), the requests work correctly most of the time, except that in about 4% of the cases the request fails and the only error I have available is what the standard .net webrequest library tells me:  "Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host." which should mean that for some reason multichain is closing down the connection while in the process of sending back a response.

I've been trying to debug this for a while without success, it may happen with any JSON RPC request and generally happens when there a bunch of requests in a short time (right now I'm doing tests with 1 second delay and it happens around the tenth consecutive request).

I don't know what's happening and how to debug it since it seems to be a problem on the side of multichain and one that happens only a small percentage of the time.

I tested on multichain community 2.0.3 but I had this happen even on 2.0.2.

What can I do to further investigate this situation?
Any idea on why it could be happening?
asked Oct 23 by mgiacomellijsb

1 Answer

0 votes

If the problem is occurring when you are sending a lot of requests at the same time, the likely explanation is that some of these requests are taking a while, and your MultiChain node is not able to answer some new incoming connections because all the RPC threads are used up. Three suggestions that might help:

  • Improve the performance of the server (maybe this is a light cloud instance?)
  • Use the rpcthreads runtime parameter to increase the number of RPC commands that can be in process in parallel.
  • Increase the timeout parameters on the client side.
answered Oct 26 by MultiChain
OK. And just to confirm you're connecting to the node's API using localhost/127.0.0.1 rather than the server's external IP address?
I can confirm that, every connection is towards localhost/127.0.0.1 and the relevant port
OK, thanks for your replies. The last thing I would ask you to try is some other method of sending the requests to the MultiChain node, instead of .NET and the webrequest library. For example you could use Apache JMeter to send a simple command to the node in a loop. That should help isolate whether the problem is in the calling library or in the operating system or node.
I tried with Advanced REST Client right on the server to send a bunch of getinfo requests to multichain and I haven't seen any issues even after repeatedly spamming the send button.
I'll try the following things next:
- try to recreate the specific request in ARC and spam it (it's a stream insertion with some values)
- try another implementation of the .net webrequest library (for instance restsharp instead system.net.http)
Just to add, what I am observing from the debug log after adding -debug=mcapi flag is the client encountered the error and MultiChain did not receive any input from the client. @mgiacomellijsb can you verify if that's the same for your case?

@mgiacomellijsb, I've also changed the .net webrequest to HttpClient but still faced with the same issue. Let me know your findings for restsharp.
...