Wednesday, March 30, 2016

Fix cURL CURLOPT_TIMEOUT_MS bug

cURL is a very useful library written in C which allows you to do a lot of network jobs like calling HTTP REST API, etc.

Recently, I need to add a timeout value to my program which uses cURL, and I found there is a convenient option called CURLOPT_TIMEOUT_MS:
https://curl.haxx.se/libcurl/c/CURLOPT_TIMEOUT_MS.html
which could add a timeout in milliseconds to a cURL connection.

But I found that whenever I set the timeout less than 1000 milliseconds, the cURL connection timeouts immediately after it is performed. After googling online for a while, I found the issue is caused by:
If libcurl is built to use the standard system name resolver, that portion of the transfer will still use full-second resolution for timeouts with a minimum timeout allowed of one second. The problem is that on Linux/Unix, when libcurl uses the standard name resolver, a SIGALRM is raised during name resolution which libcurl thinks is the timeout alarm.
One quick fix for this problem is to disable signals using CURLOPT_NOSIGNAL. For example:
curl_easy_setopt(curl, CURLOPT_NOSIGNAL, 1);
// timeout in milliseconds
// https://curl.haxx.se/libcurl/c/CURLOPT_TIMEOUT_MS.html
curl_easy_setopt(curl, CURLOPT_TIMEOUT_MS, timeoutMilliseconds);
// Perform the request, res will get the return code
res = curl_easy_perform(curl);
// Check for errors
if (res==CURLE_OPERATION_TIMEDOUT)
       throw "timeout";
else if(res != CURLE_OK)
       throw "error info: "+std::string(curl_easy_strerror(res);




Reference
https://ravidhavlesha.wordpress.com/2012/01/08/curl-timeout-problem-and-solution/

How to add timeout to an XMLRPC-C client

XMLRPC-C is very efficient if you need a remote process call protocol.

I happen to need to add timeout limit to an XMLRPC client based on C, but I went into a problem: the timeout does not work as expected.

I started with an example code in the xmlrpc-c package located at:
xmlrpc-c-1.42.99/src/examples/cpp/sample_add_client_complex.cpp (as shown in the Reference at the end of this post)
, but found that when I set the timeout less than 1000, the timeout basically does not work at all, i.e., no timeout could happen.

After looking into the codes of xmlrpc-c, I found the trick. The xmlrpc-c has a source file at:
xmlrpc-c-1.42.99/lib/curl_transport/curltransaction.c
which actually sets the timeout in the following function:
static void
setCurlTimeout(CURL *       const curlSessionP ATTR_UNUSED,
               unsigned int const timeoutMs ATTR_UNUSED) {

#if HAVE_CURL_NOSIGNAL
    unsigned int const timeoutSec = (timeoutMs + 999)/1000;

    assert((long)timeoutSec == (int)timeoutSec);
        /* Calling requirement */
    curl_easy_setopt(curlSessionP, CURLOPT_TIMEOUT, (long)timeoutSec);
#else
    /* Caller should not have called us */
    abort();
#endif
}
You can see that it actually sets the CURLOPT_TIMEOUT option of a curl connection, while CURLOPT_TIMEOUT is in seconds not milliseconds. If I originally set the timeout of xmlrpc client as 600ms, this function would convert it into 1 second, which is definitely not what I want.

One quick fix is to use my another post as follows:
static void
setCurlTimeout(CURL *       const curlSessionP ATTR_UNUSED,
               unsigned int const timeoutMs ATTR_UNUSED) {

#if HAVE_CURL_NOSIGNAL
    curl_easy_setopt(curlSessionP, CURLOPT_NOSIGNAL, 1);
    curl_easy_setopt(curlSessionP, CURLOPT_TIMEOUT_MS, timeoutMs);
#else
    /* Caller should not have called us */
    abort();
#endif
}



Reference:

// taken from xmlrpc-c-1.42.99/examples/cpp/sample_add_client_complex.cp
/*=============================================================================
                        sample_add_client_complex.cpp
===============================================================================
  This is an example of an XML-RPC client that uses XML-RPC for C/C++
  (Xmlrpc-c).

  In particular, it uses the complex lower-level interface that gives you
  lots of flexibility but requires lots of code.  Also see
  xmlrpc_sample_add_server, which does the same thing as this program,
  but with much simpler code because it uses a simpler facility of
  Xmlrpc-c.

  This program actually gains nothing from using the more difficult
  facility.  It is for demonstration purposes.
=============================================================================*/

#include
#include
#include
#include

using namespace std;

#include
#include
#include

int
main(int argc, char **) {

    if (argc-1 > 0) {
        cerr << "This program has no arguments" << endl;
        exit(1);
    }

    try {
        xmlrpc_c::clientXmlTransport_curl myTransport(
            xmlrpc_c::clientXmlTransport_curl::constrOpt()
            .timeout(10000)  // milliseconds
            .user_agent("sample_add/1.0"));

        xmlrpc_c::client_xml myClient(&myTransport);

        string const methodName("sample.add");

        xmlrpc_c::paramList sampleAddParms;
        sampleAddParms.add(xmlrpc_c::value_int(5));
        sampleAddParms.add(xmlrpc_c::value_int(7));

        xmlrpc_c::rpcPtr myRpcP(methodName, sampleAddParms);

        string const serverUrl("http://localhost:8080/RPC2");

        xmlrpc_c::carriageParm_curl0 myCarriageParm(serverUrl);

        myRpcP->call(&myClient, &myCarriageParm);

        assert(myRpcP->isFinished());

        int const sum(xmlrpc_c::value_int(myRpcP->getResult()));
            // Assume the method returned an integer; throws error if not

        cout << "Result of RPC (sum of 5 and 7): " << sum << endl;

    } catch (exception const& e) {
        cerr << "Client threw error: " << e.what() << endl;
    } catch (...) {
        cerr << "Client threw unexpected error." << endl;
    }

    return 0;
}