[SOLVED] U97 SOAP call-out to https site got error | Bugs, Issues and Errors | Forum

Avatar

Please consider registering
guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

—  Results per page  —








— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters
For a group of consecutive words like 'end of support' use Match phrase

sp_Feed Topic RSS sp_TopicIcon
[SOLVED] U97 SOAP call-out to https site got error
05 Jun 2017
12:53 pm
Avatar
Marco
Member
Forum Posts: 72
Member Since:
01 Oct 2012
sp_UserOfflineSmall Offline

Hi,

when we migrated our customer from U9401R126 to U9703G308 we got an error in SOAP call-out when try to invoke a webservice on https site.

I think that the signature is the same from U94 and U97 because we migrate the repository and build a new urr from idf with /urr command line switch.

In asn we use driver SOP U2.0 and the webservice is configured like this:

[SERVICES_EXEC]
GAC_ARTICOLI            $SOP:GAC_ARTICOLI euser=domain\user epass=password scheme=L ign=HP

 

$status and $procerror are -150 and the error in the error context variable on debug is:

ERROR -150
MNEM  
DESCRIPTION Activation error occurred
COMPONENT XSTTF10
PROCNAME LF_ARTICOLILISTA
TRIGGER OPER
LINE 19
ADDITIONAL  
  INSTANCENAME GAC_ARTICOLI
  DRV SOP
  LOCATION INVOCATION
  CODE TRANSPORT
  MESSAGE HTTP transport error
  DETAIL Request::handleRequest HTTPTransportException:HTTP transport error –
The requested URL https://xxxxxxxx   returned error:
The requested URL returned error: 411 Length Required

 

Because the webservice is available only on customer network, I create locally in my network a SOAPUI mockservice from wsdl to view a soap evelope from a uniface call to find where is the difference from U94 and U97.

U94 SOAP envelope




 true
 
 
 
 
 2017-06-01T18:07:00.00+02:00
 2017-06-05T11:34:00.00+02:00


 

U94 HTTP header
Content-Length
728
Accept-Encoding
identity
SOAPAction
"http://AvenStacWs.org/ArticoliInteresseAziendaliList"
User-Agent
CURLTransport/1.0
Accept
*/*
Content-Type
text/xml;charset=UTF-8

 

U97 SOAP envelope
 
 
  
   true
   
   
   
   
   2017-06-01T18:07:00.00+02:00
   2017-06-05T11:35:00.00+02:00
  
 


U97 HTTP header
Host
nb-019096:38080
Content-Length
727
Accept-Encoding
identity
SOAPAction
"http://AvenStacWs.org/ArticoliInteresseAziendaliList"
User-Agent
CURLTransport/1.1
Accept
*/*
Content-Type
text/xml;charset=UTF-8

 

First, I saw that the envelope was different, so I created a callback service to reproduce the same envelope, but I got error again.
Second, I saw that the User-agent identified a different cURL transport, so I decided to try replace the libcurl.dll in bin folder with U94 old version.
This is not recommended but now the SOAP call-out work fine like before migration.

Maybe the problem is on cURL dll version, is it possibile to configure the SOAP call-out to use CURL transport/1.0? Or the problem is inside a dll ?

 

Enviroment:
Platform: W2008R2
Uniface: U97G308
DBRMS: Oracle 11g

 

Thanks

Marco

05 Jun 2017
2:49 pm
Avatar
gianni
Member
Forum Posts: 331
Member Since:
01 Oct 2012
sp_UserOfflineSmall Offline

Hi Marco,

could be obvious but…considering https protocol: do you migrated related certificate as well ?

Gianni

05 Jun 2017
3:13 pm
Avatar
Marco
Member
Forum Posts: 72
Member Since:
01 Oct 2012
sp_UserOfflineSmall Offline

gianni said
Hi Marco,

could be obvious but…considering https protocol: do you migrated related certificate as well ?

Gianni  

Hi Gianni,
correct, but I haven’t certificate in U94 and even in U97.
I use ign=HP parameter on SOAP params in asn definition to remove control on there.

In my case the error is on HTTP transport, but I tried to remove ign parameter and I receved a correct certificate error.
I tried also to use a ca_bundle.crt downloaded from mozilla site, as suggested in other post, and tried different combination of ign parameter (only H, only P).

Marco

06 Jun 2017
10:25 am
Avatar
diseli
Admin
Forum Posts: 1017
Member Since:
01 Oct 2012
sp_UserOfflineSmall Offline

Hi Marco,

Do you know what web server your customer is using? And have you checked the complete response from the server in the SOAP_CALLOUT_POST operation? Maybe it contains some additional info about the error?

I’ve seen the HTTP error 411 before with (e.g.) Apache version 1.3 and chunked transfer encoding. The workaround was to use the HTTP version 1.0 instead of 1.1. It could be that the version in the User-Agent header refers to the HTTP version used by curl (“CURLTransport/1.0” and “CURLTransport/1.1”). Not sure though. Unfortunately it is not possible to specify the HTTP version the SOAP connector should used. AFAIK it will always use the internally preferred HTTP version of curl. So in case HTTP version 1.0 is required then it might be an option to call the webservice by doing the request with the curl command line tool and spawn it from Uniface (or use the OS command signature).

Hope this helps.

Kind regards,
Daniel Iseli
Uniface Technical Support

07 Jun 2017
10:52 am
Avatar
Marco
Member
Forum Posts: 72
Member Since:
01 Oct 2012
sp_UserOfflineSmall Offline

Hi Daniel,

I don’t know what web server version is on the third part (not my customer) and the SOAP_CALLOUT_POST isn’t executed after the activate command, only the SOAP_CALLOUT_PRE is executed.

So, there isn’t the solution by code? It’s not a bug? In U94 the SOAP call-out with activate works, in U97 doesn’t work anymore.

As the problem is HTTP error 411 Length Required and as both UHTTP and SOAP call-out use cURL library, is possibile that the FLAG 4 (Do not calculate the content-length of the payload itself for a POST method), visible in UHTTP operation, is set by default on SOAP call-out?

Or maybe the error is in the cURL library? In this case, replace libcurl.dll with the old version from U94 (that I tried and it works) is possible and correct?
I tried to install the new cURL library (libcurl.dll W32 edition) downloaded from cURL site, but on activate I got a -81 procerror and the error context indicates “Failure to connect to engine”.

 

Marco

07 Jun 2017
1:59 pm
Avatar
diseli
Admin
Forum Posts: 1017
Member Since:
01 Oct 2012
sp_UserOfflineSmall Offline

Marco said
Hi Daniel,

Hi Marco,

Marco said
I don’t know what web server version is on the third part (not my customer) and the SOAP_CALLOUT_POST isn’t executed after the activate command, only the SOAP_CALLOUT_PRE is executed.

Indeed, in case the returned HTTP status is 411 (or anything other than 200) then the SOAP_CALLOUT_POST operation is not activated. We could e.g. use a web proxy (like e.g. Fiddler) or the command line tool curl (Uniface 9.7.03 + G308 is e.g. using the version 7.40.0) to figure out the exact reply from the webservice.

Marco said
So, there isn’t the solution by code? It’s not a bug? In U94 the SOAP call-out with activate works, in U97 doesn’t work anymore.

Currently I can’t think of a code solution and I’m also not convinced that this is a bug. I did a quick check and my assumption seems to be correct that the Uniface 9.4 SOAP connector (i.e. libcurl) is using HTTP version 1.0 and Uniface 9.7 HTTP version 1.1. So it’s quite possible that the web server has a problem with chunked transfer encoding and that’s the reason for the HTTP error 411. A possible workaround could be to use the curl command line tool and use HTTP version 1.0 (see command line option -0, –http1.0).

Marco said
As the problem is HTTP error 411 Length Required and as both UHTTP and SOAP call-out use cURL library, is possibile that the FLAG 4 (Do not calculate the content-length of the payload itself for a POST method), visible in UHTTP operation, is set by default on SOAP call-out?

No, I don’t think so. The SOAP connector will add the Content-Length header to the request.

Marco said
Or maybe the error is in the cURL library? In this case, replace libcurl.dll with the old version from U94 (that I tried and it works) is possible and correct?
I tried to install the new cURL library (libcurl.dll W32 edition) downloaded from cURL site, but on activate I got a -81 procerror and the error context indicates “Failure to connect to engine”.

See above – I don’t think that there’s an error in the libcurl.dll. And please don’t copy the libcurl.dll from an older Uniface version or try to replace it with another curl version. The libcurl.dll used by Uniface is modified and replacing it with something else could break the SOAP connector or might make Uniface unstable.

Marco said
 

Marco  

Hope this helps.

Daniel

07 Jun 2017
3:21 pm
Avatar
rogerw
Member
Forum Posts: 268
Member Since:
01 Oct 2012
sp_UserOfflineSmall Offline

Hi,

I haven’t read your topic very carefully, but have you looked at this topic
http://unifaceinfo.com/forum/u…..vice-u9-7/

and especially this:

For some reason I had to reimport the wsdl. I see no important change in the signature exported from U9.6.04 to U9.7 and the one reimported into U9.7 ??

The above indicates that I had to reimport the wsdl with “/sti”, it wasn’t enough to migrate the signature??

And the need for using a “local” file

[SERVICES_EXEC]
smsservice=$sop:smsservice  wsdl=g:/unifaceappdir/SMSService.wsdl 

Regards RogerW.

12 Jun 2017
5:25 pm
Avatar
Marco
Member
Forum Posts: 72
Member Since:
01 Oct 2012
sp_UserOfflineSmall Offline

Hi Roger,
I tried your suggestion, but it doesn’t work, I got same error.

I tried to launch curl 7.40.0 command line and it works perfectly, so why libcurl.dll inside U97 doesn’t work?
This is the line that I run:

curl –ntlm –insecure –user userid:password –header “Content-Type: text/xml;charset=UTF-8” –header “SOAPAction:http://XXXXXXXXXXXXX/XXXXXXXXXXXXXXXXX” –data @envelope.xml https://xxxxxxxx

where envelope.xml is catched with SOAP_CALLOUT_PRE function.
I hope that the switches used are like $SOP parameter used in asn file (scheme=L ign=HP).

With verbose option, I saw that the server http protocol is 1.1.
This is the result:

  Trying 111.111.111.111…
* Connected to XXXXXXXXXXXXXXXXX (111.111.111.111) port 443 (#0)
* TLSv1.0, TLS handshake, Client hello (1):
} [199 bytes data]
* TLSv1.0, TLS handshake, Server hello (2):
{ [81 bytes data]
* TLSv1.0, TLS handshake, CERT (11):
{ [2818 bytes data]
* TLSv1.0, TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.0, TLS handshake, Client key exchange (16):
} [262 bytes data]
* TLSv1.0, TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.0, TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.0, TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.0, TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.0 / AES128-SHA
* Server certificate:
*        subject: XXXXXXXXXXXXXXXXX
*        start date: 2017-02-16 00:00:00 GMT
*        expire date: 2019-03-18 23:59:59 GMT
*        issuer: C=US; O=GeoTrust Inc.; CN=GeoTrust SSL CA – G3
*        SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Server auth using NTLM with user ‘XXXXXXXXXXX’
> POST /service.asmx HTTP/1.1
> Authorization: NTLM TlRMTVNTUAABAAAAB4IIogAAAAAAAAAAAAAAAAAAAAAGAbEdAAAADw==
> User-Agent: curl/7.40.0
> Host: XXXXXXXXXXXXXXXXX
> Accept: */*
> Content-Type: text/xml;charset=UTF-8
> SOAPAction:http://XXXXXXXXXXXXX/XXXXXXXXXXXXXXXXX
> Content-Length: 0
>
< HTTP/1.1 401 Unauthorized
< Content-Type: text/html; charset=us-ascii
< Server: Microsoft-HTTPAPI/2.0
< WWW-Authenticate: NTLM TlRMTVNTUAACAAAACAAIADgAAAAFgomi0UR/dcIDUDgAAAAAAAAAAJ4
< Date: Mon, 12 Jun 2017 14:00:40 GMT
< Content-Length: 341
<
* Ignoring the response-body
{ [341 bytes data]
* Connection #0 to host XXXXXXXXXXXXXXXXXX left intact
* Issue another request to this URL: ‘https://xxxxxxxxxxxx
* Found bundle for host XXXXXXXXXXXXXXXXX: 0x87d798
* Re-using existing connection! (#0) with host XXXXXXXXXXXXXXXXXXXXXX
* Connected to XXXXXXXXXXXXXXXX (111.111.111.111) port 443 (#0)
* Server auth using NTLM with user ‘XXXXXXXXXXXXXXXX’
> POST /service.asmx HTTP/1.1
> Authorization: NTLM TlRMTVNTUAADAAAAGAAYAIgAAAAmASYBoAAAAAgACABYAAAAHAAcAGAAAA
> User-Agent: curl/7.40.0
> Host: XXXXXXXXXXXXXXXXXXXXXX
> Accept: */*
> Content-Type: text/xml;charset=UTF-8
> SOAPAction:http://XXXXXXXXXXXXX/XXXXXXXXXXXXXXXXX
> Content-Length: 713
>
} [713 bytes data]
* upload completely sent off: 713 out of 713 bytes
< HTTP/1.1 200 OK
< Cache-Control: private, max-age=0
< Content-Type: text/xml; charset=utf-8
< Server: Microsoft-IIS/7.5
< X-AspNet-Version: 2.0.50727
< Persistent-Auth: true
< X-Powered-By: ASP.NET
< Date: Mon, 12 Jun 2017 14:00:42 GMT
< Content-Length: 703748
<
{ [16128 bytes data]
* Connection #0 to host XXXXXXXXXXXXXXXXX left intact

 

 Marco

14 Jun 2017
5:40 pm
Avatar
diseli
Admin
Forum Posts: 1017
Member Since:
01 Oct 2012
sp_UserOfflineSmall Offline

Hi Marco,

Thanks for the additional info. This is really helpful.

From the curl tracing I can see that the Web Service is Microsoft IIS 7.5. I did a quick test here on one of our Windows Test-Servers and I can see the problem now as well. It seems that when the SOAP connector is sending the initial request with the NTLM authorization then the Content-Length header is not set (and this seems to trigger the HTTP error 411). It looks like that this issue was introduced after we’ve upgraded the curl version used by Uniface to 7.40.0 (with the version 9.6.06 patch X604 and version 9.7.01). Before the curl upgrade the SOAP connector sets the Content-Length header to 0 (“Content-Length: 0”) and after the upgrade the header is omitted. I’ll discuss this with the lab and shall keep you posted.

The good news is that I’ve found a workaround: when you set the scheme connector option to LN (NTLM and Negotiate) then the request should work (again) correctly. E.g.

[SERVICES_EXEC]
GAC_ARTICOLI            $SOP:GAC_ARTICOLI euser=domain\user epass=password scheme=LN ign=HP

 

Hope this helps.

Daniel

15 Jun 2017
3:21 pm
Avatar
Marco
Member
Forum Posts: 72
Member Since:
01 Oct 2012
sp_UserOfflineSmall Offline

Hi Daniel,
I tried the workaround and now SOAP call-out works correctly again.

Thanks

Marco

15 Jun 2017
3:48 pm
Avatar
diseli
Admin
Forum Posts: 1017
Member Since:
01 Oct 2012
sp_UserOfflineSmall Offline

Marco said
Hi Daniel,
I tried the workaround and now SOAP call-out works correctly again.

Thanks

Marco  

Thanks, Marco.

It’s good to hear that the workaround is solving the problem.

In the meantime I had a chat with one of the engineers in the lab. It seems that curl has a problem when the Context-Length header is explicitly set (which the SOAP connector does for all requests) and NTLM authentication is used. curl also seems to add the mentioned header (i.e. Content-Length: 0) and when it already has been set by the calling code (i.e. the SOAP connector) then this will result in the Context-Length header being removed from the request. This was not an issue with older curl versions (at least before version 7.40). The workaround for this issue is to omit the Content-Length header from the request (since it automatically will be added by curl). I’ll create a new bug report for this issue. But I don’t think that this will be resolved quickly, since there is an easy workaround for this issue (i.e. set scheme=LN).

Hope this helps. If you have any further questions or concerns regarding this issue then let me know.

Daniel

19 Jun 2017
11:31 am
Avatar
diseli
Admin
Forum Posts: 1017
Member Since:
01 Oct 2012
sp_UserOfflineSmall Offline

Just a quick update: I’ve created in the meantime, as promised, a new BUG report for this problem:

  • BUG: 31620 – SOP U2.0: WS Call-Out fails when authentication scheme NTLM is used

This issue will be added to the maintenance backlog with a low priority. In case this problem is time critical for someone then please log a call with support.

Hope this helps.

Daniel

07 Mar 2018
1:58 pm
Avatar
diseli
Admin
Forum Posts: 1017
Member Since:
01 Oct 2012
sp_UserOfflineSmall Offline

Just another update: the BUG#31620 just has been fixed with the patch G413.

Hope this helps.

Daniel

Forum Timezone: Europe/Amsterdam

Most Users Ever Online: 131

Currently Online:
27 Guest(s)

Currently Browsing this Page:
1 Guest(s)

Top Posters:

ulrich-merkel: 1840

Iain Sharp: 663

Theo Neeskens: 366

gianni: 331

istiller: 285

rogerw: 268

Knut: 219

lalitpct: 197

Arjen van Vliet: 182

sochaz: 173

Member Stats:

Guest Posters: 3

Members: 9740

Moderators: 0

Admins: 8

Forum Stats:

Groups: 1

Forums: 62

Topics: 2270

Posts: 9824

Newest Members:

agrohimujg, Ramhaturiff, best_auto, kings1, skorp0883, oraha, Anitastync, marina1993m, CharlesRip, Angiefoxlome

Administrators: admin: 23, Adrian Gosbell: 318, diseli: 1017, Bob Maier: 3, Nico Peereboom: 80, Michael Rabone: 4, richiet: 406, JanCees: 34