MultiChain Developer Q&A - Recent questions tagged peers

In a roundrobin - if I set mineemptyrounds to 1 what will happen?

Sat, 15 May 2021 10:53:16 +0000

This doesnt make sense to me, could please explain what is expected behaviour?

3 nodes
2 nodes (A,B) are mineemptyrounds 0
1 node (C) is mineemptyrounds 1

A,B - nothing to mine - does it wait?
When it's the turn of C, does it simple adopy mineemptyround and pass back to A?

Nodes not mining and -1 in synced headers and synced blocks

Sun, 29 Nov 2020 07:55:29 +0000

Hello,

I checked the getpeerinfo command on a blockchain running from past 8 months I found the synced blocks and synced headers of one of the peers is -1 and the another peer is not syncing the mempool. The mining seems to have paused as well, leaving a lot of transactions in the mempool which are not being mined and synced.

{
        "id" : 16,
        "addr" : "10.0.4.238:57148",
        "addrlocal" : "11.0.2.19:6004",
        "services" : "0000000000000001",
        "lastsend" : 1606634315,
        "lastrecv" : 1606634315,
        "bytessent" : 64775843,
        "bytesrecv" : 58707709,
        "conntime" : 1597433750,
        "pingtime" : 0.036274,
        "version" : 70002,
        "subver" : "/MultiChain:0.2.0.10/",
        "handshakelocal" : "1K73Tko1w9ZVPaSL5L5RHFEBnbNVWTHB5c8owZ",
        "handshake" : "1WmQoWUM5GjErTH1KnjQjDSThGEFapx6GMqTGQ",
        "inbound" : true,
        "startingheight" : 6738,
        "banscore" : 0,
        "synced_headers" : -1,
        "synced_blocks" : -1,
        "inflight" : [
        ],
        "whitelisted" : false
    },
    {
        "id" : 17,
        "addr" : "10.0.2.218:41652",
        "addrlocal" : "11.0.2.19:6004",
        "services" : "0000000000000001",
        "lastsend" : 1606634313,
        "lastrecv" : 1606634313,
        "bytessent" : 6121210,
        "bytesrecv" : 54235,
        "conntime" : 1606633541,
        "pingtime" : 0.078501,
        "version" : 70002,
        "subver" : "/MultiChain:0.2.0.10/",
        "handshakelocal" : "1K73Tko1w9ZVPaSL5L5RHFEBnbNVWTHB5c8owZ",
        "handshake" : "1UgYYerpfmaV5R1sMpWSci6ymP4HD25G2FPHAs",
        "inbound" : true,
        "startingheight" : 6738,
        "banscore" : 0,
        "synced_headers" : -1,
        "synced_blocks" : -1,
        "inflight" : [
        ],
        "whitelisted" : false
    }

There are a lot of transactions in the mempool of this node which are not getting cleared, the debug.log ends up with

2020-11-29 07:32:08 MultiChainMiner: Block Found - 00bae732a4e336dce3d98613735b74d0cf3f9ccd74a166313bffa2972b556713, prev: 00ede3b542cf0a2ec323bbb547fe7d7c58e56e1969f11970c8f118395e914428, height: 6739, txs: 1312
2020-11-29 07:32:08 VerifyBlockMiner: Permission denied for miner 1K73Tko1w9ZVPaSL5L5RHFEBnbNVWTHB5c8owZ received in block signature
2020-11-29 07:32:08 VerifyBlockMiner: Block 00bae732a4e336dce3d98613735b74d0cf3f9ccd74a166313bffa2972b556713 (height 6739) miner verification failed
2020-11-29 07:32:08 ERROR: ProcessNewBlock : AcceptBlock FAILED
2020-11-29 07:32:08 ERROR: MultiChainMiner : ProcessNewBlock, block not accepted
2020-11-29 07:32:10 MultiChainMiner: Block Found - 00eae7e02fe799d07e969164b2b098573d6da5efa11d362483fcd007aa71b740, prev: 00ede3b542cf0a2ec323bbb547fe7d7c58e56e1969f11970c8f118395e914428, height: 6739, txs: 1312
2020-11-29 07:32:11 VerifyBlockMiner: Permission denied for miner 1K73Tko1w9ZVPaSL5L5RHFEBnbNVWTHB5c8owZ received in block signature
2020-11-29 07:32:11 VerifyBlockMiner: Block 00eae7e02fe799d07e969164b2b098573d6da5efa11d362483fcd007aa71b740 (height 6739) miner verification failed
2020-11-29 07:32:11 ERROR: ProcessNewBlock : AcceptBlock FAILED
2020-11-29 07:32:11 ERROR: MultiChainMiner : ProcessNewBlock, block not accepted

all the three nodes have mining permission, with mining diversity set to 0.66.

Tried stopping and restarting the nodes except the one with mempool, but nothing different happened.

Thank You

Random ports in peer handshake

Sun, 19 Jul 2020 08:22:39 +0000

While looking at the output of getpeerinfo api call, there are random ports on addr and addr local fields. I have set the default network and RPC ports.

I have set the default network ports to be 6001 for both the nodes and connected the daemon by on the same port.

here is an example

[
    {
        "id" : 3,
        "addr" : "11.0.2.19:6001",
        "addrlocal" : "10.0.2.218:40384",
        "services" : "0000000000000001",
        "lastsend" : 1595145954,
        "lastrecv" : 1595145954,
        "bytessent" : 33682716,
        "bytesrecv" : 68302709,
        "conntime" : 1592493648,
        "pingtime" : 0.100124,
        "version" : 70002,
        "subver" : "/MultiChain:0.2.0.10/",
        "handshakelocal" : "1QEXPURG6es3LhbUNux2DELwwJyB1Jqvd9xnV4",
        "handshake" : "14MwcB2MQxLT9bcfUJR5vRrnJdJ8RWAeGGnVV5",
        "inbound" : false,
        "startingheight" : 2734,
        "banscore" : 0,
        "synced_headers" : 8676,
        "synced_blocks" : 8676,
        "inflight" : [
        ],
        "whitelisted" : false
    },
    {
        "id" : 28,
        "addr" : "10.0.4.238:59848",
        "addrlocal" : "10.0.2.218:6001",
        "services" : "0000000000000001",
        "lastsend" : 1595145954,
        "lastrecv" : 1595145953,
        "bytessent" : 6403475,
        "bytesrecv" : 356378,
        "conntime" : 1595140368,
        "pingtime" : 0.05432,
        "version" : 70002,
        "subver" : "/MultiChain:0.2.0.10/",
        "handshakelocal" : "1QEXPURG6es3LhbUNux2DELwwJyB1Jqvd9xnV4",
        "handshake" : "1ZqTLXmJHvWrvpTniZuT4vusFBS1XUqWrZxkLL",
        "inbound" : true,
        "startingheight" : 87,
        "banscore" : 0,
        "synced_headers" : 8675,
        "synced_blocks" : 8675,
        "inflight" : [
        ],
        "whitelisted" : false
    }
]

Is this normal behavior? Will it cause any problems if there are firewall rules in place that only allow communication through the default network port?

Thanks

Nodes not mining

Sat, 18 Jul 2020 12:12:52 +0000

The mining in a blockchain has unexpectedly stopped, I had a blockchain running for about a couple of months, the blockchain has three nodes all with mining permissions, and mining diversity set to 0.66.

The blockchain was working fine but today I noticed excess memory consumption by the blockchain, on closer inspection I found that the mining has stopped about a month ago, and the mempool is filled with about 4000 transactions.

All three nodes are still connected and broadcasting new transactions to each other but none of the transactions is getting mined, the last block was mined on 24th June 2020.

In getinfo api call, the mining paused parameter is false.

This is the debug log, which also shows something different happening after 24th june 2020

2020-06-24 23:32:30 ResendWalletTransactions()
2020-06-24 23:32:39 UpdateTip: new best=00930c22e713129df7458d6c5e20ca3de5c17ec4ff1ba0b175f489734c4f73b2 height=31160 log2_work=22.927454 tx=34372 date=2020-06-24 23:31:24 progress=1.000000 cache=0
2020-06-24 23:32:41 MultiChainMiner: Block Found - 002120d5c508d36fb47ebc1ebf5f63fa2a259288acc299834d41d2c5b0820b70, prev: 00930c22e713129df7458d6c5e20ca3de5c17ec4ff1ba0b175f489734c4f73b2, height: 31161, txs: 1
2020-06-24 23:32:41 UpdateTip: new best=002120d5c508d36fb47ebc1ebf5f63fa2a259288acc299834d41d2c5b0820b70 height=31161 log2_work=22.9275 tx=34373 date=2020-06-24 23:32:41 progress=1.000000 cache=0
2020-06-24 23:33:03 ResendWalletTransactions()
2020-06-26 16:18:21 CommitTransaction: 1f1ab826af80026512b92be1a9c515722405e2788e5cac536ab3b94a761e3a09, vin: 1, vout: 2
2020-06-26 16:25:02 CommitTransaction: b9f51c3f27fdb2039a5f6dbb3567206dbdb5d0dce0f5416fa0a1ce2b9330b05e, vin: 1, vout: 2
2020-06-26 16:25:12 CommitTransaction: 38083a29ced1dbf2db12f18241f8ef9ebf10d04252af0020f2b8b36df7d35323, vin: 1, vout: 2
2020-06-26 16:25:22 CommitTransaction: bd189c83a98aebd1e06013951fdd69f62ff899646b8b84be84b68b28e482748e, vin: 1, vout: 2
2020-06-26 16:25:52 CommitTransaction: 425b2ca954dc1ab436e891b2daafa7217034f89183915e531eb4013f161a02a8, vin: 1, vout: 2
2020-06-26 16:26:02 CommitTransaction: 961a228bd581f965ca6060486d49b234e14b921ac37aacc8ffc897478297f281, vin: 1, vout: 2
2020-06-26 16:26:12 CommitTransaction: 639bda696c2ae49fe476c7a4ad608610659a20fdad3e035e45be897b30270896, vin: 1, vout: 2
2020-06-26 16:26:22 CommitTransaction: b9308a0699474402195b9e6a39c446c304bf8cbd0a6e070f43a688941efaba8f, vin: 1, vout: 2
2020-06-26 16:27:52 CommitTransaction: eae40ed402ae2a5e6402b49abc6798031ecab43e7d094416e299cfcb785782d8, vin: 1, vout: 2
2020-06-26 16:28:02 CommitTransaction: da7d7ab2668fda67901283335e4db1c412e60dbdcd71bba33100471197364e28, vin: 1, vout: 2
2020-06-26 16:28:12 CommitTransaction: f6335a9cc5b70201217f4d094c481cbaae79e129146a4af59211690d01c70587, vin: 1, vout: 2
2020-06-26 16:28:22 CommitTransaction: 71ddc56bb11311be177d0bdcb3b7bcdfb8c2a00df7a9eb4dc41ae8a486c4d166, vin: 1, vout: 2
2020-06-26 16:28:32 CommitTransaction: 45d3ea3644fe15aff4a7f1597af769a9e15c0bfbef77d6ea9b23627ba840ca20, vin: 1, vout: 2
2020-06-26 16:30:22 CommitTransaction:

.........

2020-07-05 15:32:39 socket sending timeout: 61s
2020-07-05 15:35:15 mchn: Sending minimal parameter set to 10.0.2.218:47216
2020-07-05 15:35:15 receive version message: /MultiChain:0.2.0.10/: version 70002, blocks=31161, us=11.0.2.19:6011, peer=44
2020-07-05 15:35:15 mchn: Connection from 1LWFDzTX9GHwHdpx8zBLExZbwAevCGC8s3EuVS received on peer=44 in verack
2020-07-05 15:35:15 mchn: Connection from 1LWFDzTX9GHwHdpx8zBLExZbwAevCGC8s3EuVS received on peer=44 in verackack (10.0.2.218:47216)
2020-07-05 15:35:15 mchn: Parameter set from peer=44 verified
2020-07-05 15:35:16 mchn: Synced with node 44 on block 31161 - requesting mempool
2020-07-05 15:35:16 mchn: Sending minimal parameter set to 10.0.4.238:6011
2020-07-05 15:35:16 receive version message: /MultiChain:0.2.0.10/: version 70002, blocks=31160, us=11.0.2.19:58198, peer=45
2020-07-05 15:35:16 mchn: Connection from 149kFCMKAenKXZAcENGLD7FMB5BvuXt1JhHGSb received on peer=45 in verack
2020-07-05 15:35:16 mchn: Connection from 149kFCMKAenKXZAcENGLD7FMB5BvuXt1JhHGSb received on peer=45 in verackack (10.0.4.238:6011)
2020-07-05 15:35:16 mchn: Parameter set from peer=45 verified
2020-07-05 19:31:50 CommitTransaction: 0213d891653686ee1dc9ba5153077aa73c7734abe8cd90a424a3f5ce07b71051, vin: 1, vout: 2
2020-07-05 19:32:09 CommitTransaction: e617efe2efee075b85fdcd0ce6ac641a22e19e5e4f49cb8f64c7c2542f2a95d0, vin: 1, vout: 2
2020-07-05 19:35:08 CommitTransaction: 61ee58ce6c5fd3b1e2a2a8e858709d6196c5f832df7dc090d494f8f5517d6937, vin: 1, vout: 2
2020-07-06 04:55:41 CommitTransaction: 3a02331bea3751f9ccaf3853da3d0bbd45ae066b51885bd07a2518eedb032bdd, vin: 1, vout: 2
2020-07-06 05:11:08 CommitTransaction: cb20a200e4be2321faf58ebcfddcdd6d6de4745f0ae08cb7d4581c410e56dd43, vin: 1, vout: 2
2020-07-06 05:14:39 CommitTransaction: 1b695abd3ec00e89c84434486ab8f3542b5a7f1e4985a27fd48f2f2b87497364, vin: 1, vout: 2
2020-07-06 05:26:15 CommitTransaction: 4c732e38089e3006fe91556bb439033962bd3ed9bae38bfb45f151656de95d0c, vin: 1, vout: 2
2020-07-06 05:29:38 CommitTransaction: acfd840431885ea2b2c4744dbc5380e91a50890128e164ac2a5d9175ec25f5b8, vin: 1, vout: 2
2020-07-06 05:32:59 CommitTransaction: 405f1729041503fa7a6af5d86851de1fb80d60a05040e4f5b49a0551202d7023, vin: 1, vout: 2
2020-07-06 05:50:54 CommitTransaction: 34f0f285471d4b1e0ace1b42b94df6992ab7f2130b1e7ceb7a80d9018e826097, vin: 1, vout: 2
2020-07-06 06:06:15 CommitTransaction: 36b7c44a74619a98acbd1018179b26c8aa47b66edc03e6e76bf38ff316372d9d, vin: 1, vout: 2
2020-07-06 06:06:32 CommitTransaction: 257f061dc05453f11f1ce6116d29df8336893e220b006f4d64b7b9f0bc5c97be, vin: 1, vout: 2
2020-07-06 06:19:55 CommitTransaction: 71596557511585e33d0809f3182165232e7a4facb34c8a35cdf5e437000e1f98, vin: 1, vout: 2
2020-07-06 06:21:25 CommitTransaction: 424815aa076186c4fd8ca66412371ebb37657513858b85b6314dbfd52745834d, vin: 1, vout: 2
2020-07-06 06:24:13 CommitTransaction: bb770909698bf71d99f2e02a677bd58748878b7f395f119156cd7f7088a93b3e, vin: 1, vout: 2
2020-07-06 06:27:54 CommitTransaction: 080825c10bd21852a35f21c5246ef39e2e15ab5c4e969993cb37c398b5879cf8, vin: 1, vout: 2
2020-07-06 06:45:58 CommitTransaction: 0c0837454310870bda106667b26951d9663517697f3640eab1ce526a7f6f7437, vin: 1, vout: 2
2020-07-06 06:57:58 CommitTransaction: 1fc900bbb1b451d086d245390dc105f1e63462c0c1a7d94044d95be66b370c8f, vin: 1, vout: 2
2020-07-06 06:58:09 CommitTransaction: 240a1a82dcd2ec4b224854ddbfcaba036e1992df3d8543ef409df4908ab067b6, vin: 1, vout: 2
2020-07-06 08:41:09 CommitTransaction: 433eea579db4969491960d4856c1d7938f042d9330dc6766c77ba937cb622876, vin: 1, vout: 2
2020-07-06 08:41:29 CommitTransaction: e240a48e37eb9bbac01048bb39e10df98ae2bfb8e73e194f21604c75d3f2dde1, vin: 1, vout: 2
2020-07-06 08:45:55 CommitTransaction: 4ac900ac0137b28767e0ad708ef6e335b9e1b83b3e825390eb98ed7a8e5f527e, vin: 1, vout: 2
2020-07-06 08:47:44 CommitTransaction: 7b309d628c715f631d94fec1c50c5732f3bf4423c6d9615080865624d24c559e, vin: 1, vout: 2
2020-07-06 08:48:30 CommitTransaction: 7bd43c5e4ff5152a7195ba9d0e288a566c4e80041ed31074fa5613499e231192, vin: 1, vout: 2
2020-07-06 08:49:13 CommitTransaction: 411ec0d859f8e3183e3b54782f4214e2d4e26c6ea8806247f20932f864ed4570, vin: 1, vout: 2
2020-07-06 08:49:34 CommitTransaction: d81d15d5c67b3ecf190271121491871b5226278a40bf7cf485be4907d8393d8b, vin: 1, vout: 2

Did not post the full log as the logs are quite a lot. The logs end with commit transaction

Not able to find the exact cause of the problem.

Thanks

Connection failure between nodes?

Wed, 04 Mar 2020 15:36:13 +0000

What happens if the connection between a node and the network is broken.

However, the node does not notice the disconnection and continues working for 2 day. After 48 hours the node can reconnect to the network. What happens now with the transactions and different lengths of the blockchain?

peer network with different startingheight value

Sat, 28 Sep 2019 15:43:07 +0000

Dear Team,

I have a network composed by 4 nodes, one seed node and other 3 nodes connected to it. I checked they are in sync with the <getblockchaininfo> command. All of them show the same values for the following parameters:
"blocks" : 477,
"headers" : 477,
"bestblockhash" : "00110f363f98ff344e93d5491b79644783abd2f58a7b257540be7c416596a562"

However, if I log in in the seed node and I send the command <getpeerinfo>, I get the list of the 3 peers with a different value for one of them:
[
{
"id" : 1,
"startingheight" : 462,
"banscore" : 0,
"synced_headers" : 476,
"synced_blocks" : 476,
},
{
"id" : 3,
"startingheight" : 477,
"banscore" : 0,
"synced_headers" : 477,
"synced_blocks" : 477,
},
{
"id" : 4,
"startingheight" : 477,
"banscore" : 0,
"synced_headers" : 477,
"synced_blocks" : 477,
}
]

Could you kindly help me in understanding? Thanks, Fabio

Creating manually conection between 2 nodes

Wed, 28 Aug 2019 08:08:44 +0000

I have a network of 6 node, some of them doesent create a peer between them.

How do I create the conection ?

High CPU usage when attempting to connect to unavailable nodes

Wed, 26 Dec 2018 22:21:43 +0000

I believe I have identified a bug with the peer reconnection code. I have a node that has done nothing for the last month. A month ago I generated a few blocks and connected to it from a couple other nodes, but then shut those other nodes down. The main node has been attempting to reconnect to those 2 nodes for the last month, and spinning more and more of the CPU as each day goes by until it's using about 25% of the instance's CPU now. I got an alert about low CPU credits this morning and found this to be the problem when I investigated.

When I run the node with `-debug` I see these lines repeated over and over again:

net: Trying to connect to 123.123.123.123:6700
  net: trying connection 123.123.123.123:6700 lastseen=627.1hrs
  net: connect() to 123.123.123.123:6700 failed after select(): Connection refused (111)
  net: Connection not established
  net: Trying to connect to 192.168.1.49:6700
  net: trying connection 192.168.1.49:6700 lastseen=627.1hrs
  net: connection to 192.168.1.49:6700 timeout
  net: Connection not established

If I start the node in either `-offline` mode, or if I delete the `peers.dat` file before starting, it does not exhibit this behavior, and sits at 0% CPU like I expect for a node that only has 50 blocks, is not mining and has no actual peers.

This is with versions 2.0a5, 2.0a6, 2.0a7 and 2.0b1.

I also tested copying the .multichain directory to another instance on a newer version of Ubuntu (bionic instead of xenial), and a non-vm computer, and I get the same behavior.

This is a graph showing how it has escalated over the last month.

How long will it wait before it stops trying to connect to these dead nodes? And why is there no way to view what is in the peers.dat file, and to clean them up? I've searched around and not found any definitive answers to how this system works. (or is supposed to work.)

Thanks!

Nate

Node Discovery

Thu, 21 Jun 2018 21:34:53 +0000

Guys

Do you have any more information on how nodes discover peers either via other connected peers or using addnode?

For context we are running multiChain nodes within Kubernetes and we need to ensure nodes don't get orphaned from each other on a restart which can cause the nodes to be assigned new IP addresses. Because of the way Kubernetes works, we can fix the DNS names of the nodes and we experimented with added these across each node using the addnode api call, but these don't seem to persist across restarts.

At the minute we might need to execute addnode at startup of each node, but wanted to check if this was expected behaviour of addnode and any more information on how peer discovery works and any guidelines you might have to ensure the network remains connected across node restarts and changes of node IP addresses?

Cheers

Marty

How nodes connect and link between nodes

Fri, 04 Aug 2017 10:28:26 +0000

I dont think i understand the link between nodes correctly.

Just some background info on what im trying to do:

If i have 3 nodes and all 3 are equal peers.

I create addresses on one of these nodes that i treat like a 'master node' and all my assets are on this 'master node' as well.I move my assets between these addresses and assets on this 'master node'. Should this 'master node' crash or die or be hacked. I cant use one of the peers to continue the work that was done by my 'master node' as the peers do not posses the private/public key pairs that the 'master node' had control over.

I am simulating a wallet scenario like blockchain.info where i have one 'master node' who knows all addresses and each user gets a new address from the 'master node'. Then i issue assets from the 'master node' to these addresses on this node.

I only use the peers to mine and to sync data between them.

I can use importaddress but this only gives me a watch on those addresses but not the keys. I understand this is a security mechanism but I'm not sure how the nodes are all equal and they replicate the data/address/balances and the blocks among them.

So my question is,

a) What do i do in a case where my 'master node' is no longer usable in any circumstance be it a hack/crash/corruption?

b) Why cant i just fallback onto one of the peers and let them continue the work the 'master node' was doing?

Instead of having to be forced to either keep a hard backup of that node or recreate all blocks for that 'master node'.

I know there is no concept of a master node in block/multichain but I'm not sure how else to refer to this node which knows this type of information.

GetPeers doesnt return nodes that have No Permissions

Thu, 20 Jul 2017 13:38:18 +0000

If node A, as been revoked of all permissions, but I still wish to have that node listed as "No Permissions" - so that permissions can then be granted at another time, I was hope to get a list of all nodes, irrespective of permissions. Am I missing a call or parameter that will return the full list of peers, connected or not?

Managing public / private IPs for peering

Mon, 24 Apr 2017 10:35:20 +0000

Just a question, but if I have multiple multichain nodes that all reside within a reastricted network, and then attach a peer coming in via an external IP that only has access to one designated multichain node, then as far as available peers go, my new external node will try and connect to other peers that are not reachable from the network zone it's in.

Is there a way to manage what nodes are available as peers and additionally is there a way to configure public endpoints on a peer to peer basis that could be used for peering once it's determined the private IPs are not reachable....

Hope this makes sense, but looking at a potential deployment scenario which will have a mix of nodes some being public and others private and how to manage the comms between those clusters?

Cheers

Marty