High amount of timeouts when forwarding payments

General Questions
1

Hi,
I would like to ask for advice. We have a connector connected to an ILP network that forwards packets of money from one payment pointer to another. For this, we use web-monetization-access library from Github. We have encountered a problem wherein many cases said packets are not forwarded. It seems that the incoming packets are fine. When we get them, we’ll forward them to the appropriate payment pointer.
The connection stream is expected to emit an outgoing money event after the packets are sent. Everything works fine when the connector starts, but after a while, payments will stop being sent and outgoing money is no longer emitted.
Subsequently, one of these two cases occur:

  1. The first, less common, case is that the event is recalled after a few minutes with an accumulated amount. It appears to occur during initial outages.

  2. The second, the more common, case is that we get a timeout error:
    Remote connection error. Code: InternalError, message: 4231604317 timed out

This problem arises on a particular connection, and if the payment is sent to another payment pointer, the dispatch works, at least until the same thing happens. Sometimes it recovers after a while, but rarely. I also have debug logs from ILP, but I’m not very familiar with these.

We use version 22.5.0 and our connector config is:

'use strict'

const path = require('path')

const parentConnectors =
{
  "ilsp": {
    "relation": "parent",
    "plugin": "ilp-plugin-http",
    "sendRoutes": false,
    "receiveRoutes": false,
    "assetCode": "XRP",
    "assetScale": 9,
    "maxPacketAmount": "1000000000",
    "rateLimit": {
      "refillCount": 1000000000
    },
    "options": {
      "incoming": {
        "port": <port>,
        "secretToken": "<secretToken>"
      },
      "outgoing": {
        "url": "<outgoingUrl>",
        "secretToken": "<secretToken>",
        "http2": true,
        "http2MaxRequestsPerSession": 900,
        "name": "<name>"
      }
    }
  },
  "child": {
    "relation": "child",
    "plugin": "ilp-plugin-mini-accounts",
    "assetCode": "XRP",
    "assetScale": 9,
    "rateLimit": {
      "refillCount": 1000000000
    },
    "options": {
      "port": <port>
    }
  }
}

const connectorApp = {
  name: 'connector',
  env: {
    CONNECTOR_ENV: "production",
    CONNECTOR_BACKEND: "ecb-plus-xrp",
    CONNECTOR_SPREAD: "0",
    CONNECTOR_STORE_PATH: "<storePath>",
    CONNECTOR_ADMIN_API: "true",
    CONNECTOR_ADMIN_API_PORT: "<port>",
    DEBUG: "ilp*",
    CONNECTOR_ACCOUNTS: JSON.stringify(parentConnectors)
  },
  interpreter: '/usr/local/n/versions/node/10.16.2/bin/node',
  script: path.resolve(process.execPath, '../../lib/node_modules/ilp-connector/src/index.js')
}

module.exports = { apps: [ connectorApp ] }

Is there any way I can find out why payments aren’t sent?
Or can we somehow find out if this is happening on our side and we have to fix something?

I would really appreciate your help, thank you.

2

Hi @romcok - sorry to hear you’re having problems here. I’m not familiar enough with the inner workings of the Javascript connector to give you any immediately useful here (I would be more useful with Java Connector questions though since I help maintain that implementation), but if this is still a problem, you might find more response on the ILP slack, or even consider filing an issue in the JS Connector Github project.

If I’m reading your report correctly, it looks like you have only a single outgoing connection (id=ilsp). If true, sending to a different payment pointer will still send traffic out through your Connector’s ilsp link, which is why you might see the same problem across two different payment pointer destinations – the actual HTTP path is probably the same, and I suspect this is where problems are occuring.

Without knowing who your upstream ILSP is, I would look there first – for example, if you have access to that system’s logs, you might be able to see if that HTTP server is having trouble servicing all the traffic you’re sending it (?)

Let me know what you find out there, or if you have more info – I can try my best to help figure out what’s going wrong.