Live tests fail for v6.14 on Windows 10 Strawberry Perl 5.24.1.1 behind HPE firewall

[pwr22]

Installing 6.13 works fine. I am behind a restrictive outbound firewall with a proxy for external access, http_proxy and https_proxy are set correctly

Perl -V

Summary of my perl5 (revision 5 version 24 subversion 1) configuration:

  Platform:
    osname=MSWin32, osvers=6.3, archname=MSWin32-x64-multi-thread
    uname='Win32 strawberry-perl 5.24.1.1 #1 Mon Jan 16 02:00:29 2017 x64'
    config_args='undef'
    hint=recommended, useposix=true, d_sigaction=undef
    useithreads=define, usemultiplicity=define
    use64bitint=define, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags =' -s -O2 -DWIN32 -DWIN64 -DCONSERVATIVE  -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -fwrapv -fno-strict-aliasing -mms-bitfields',
    optimize='-s -O2',
    cppflags='-DWIN32'
    ccversion='', gccversion='4.9.2', gccosandvers=''
    intsize=4, longsize=4, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
    ivtype='long long', ivsize=8, nvtype='double', nvsize=8, Off_t='long long', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='g++', ldflags ='-s -L"C:\STRAWB~1\perl\lib\CORE" -L"C:\STRAWB~1\c\lib"'
    libpth=C:\STRAWB~1\c\lib C:\STRAWB~1\c\x86_64-w64-mingw32\lib C:\STRAWB~1\c\lib\gcc\x86_64-w64-mingw32\4.9.2
    libs=-lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    perllibs=-lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32
    libc=, so=dll, useshrplib=true, libperl=libperl524.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_win32.xs, dlext=xs.dll, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags='-mdll -s -L"C:\STRAWB~1\perl\lib\CORE" -L"C:\STRAWB~1\c\lib"'


Characteristics of this binary (from libperl):
  Compile-time options: HAS_TIMES HAVE_INTERP_INTERN MULTIPLICITY
                        PERLIO_LAYERS PERL_COPY_ON_WRITE
                        PERL_DONT_CREATE_GVSV
                        PERL_HASH_FUNC_ONE_AT_A_TIME_HARD
                        PERL_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS
                        PERL_MALLOC_WRAP PERL_PRESERVE_IVUV USE_64_BIT_INT
                        USE_ITHREADS USE_LARGE_FILES USE_LOCALE
                        USE_LOCALE_COLLATE USE_LOCALE_CTYPE
                        USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO
                        USE_PERL_ATOF
  Built under MSWin32
  Compiled at Jan 16 2017 02:12:13
  @INC:
    C:/Strawberry/perl/site/lib/MSWin32-x64-multi-thread
    C:/Strawberry/perl/site/lib
    C:/Strawberry/perl/vendor/lib
    C:/Strawberry/perl/lib
    .

cpanm Net::HTTP@6.14

cpanm (App::cpanminus) 1.7043 on perl 5.024001 built for MSWin32-x64-multi-thread
Work directory is C:\Users\rpeter/.cpanm/work/1493136176.12116
You have make C:\Strawberry\c\bin\dmake.exe
You have LWP 6.26
Falling back to Archive::Tar 2.24
Searching Net::HTTP (== 6.14) on cpanmetadb ...
--> Working on Net::HTTP
Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/Net-HTTP-6.14.tar.gz
-> OK
Unpacking Net-HTTP-6.14.tar.gz
Entering Net-HTTP-6.14
Checking configure dependencies from META.json
Checking if you have ExtUtils::MakeMaker 6.58 ... Yes (7.24)
Configuring Net-HTTP-6.14
Running Makefile.PL
Checking if your kit is complete...
Looks good
Generating a dmake-style Makefile
Writing Makefile for Net::HTTP
Writing MYMETA.yml and MYMETA.json
-> OK
Checking dependencies from MYMETA.json ...
Checking if you have IO::Socket::INET 0 ... Yes (1.35)
Checking if you have IO::Uncompress::Gunzip 0 ... Yes (2.070)
Checking if you have warnings 0 ... Yes (1.36)
Checking if you have Data::Dumper 0 ... Yes (2.161)
Checking if you have Test::More 0 ... Yes (1.302083)
Checking if you have strict 0 ... Yes (1.11)
Checking if you have File::Spec 0 ... Yes (3.63_01)
Checking if you have base 0 ... Yes (2.23)
Checking if you have Carp 0 ... Yes (1.40)
Checking if you have Socket 0 ... Yes (2.024)
Checking if you have Compress::Raw::Zlib 0 ... Yes (2.071)
Checking if you have IO::Select 0 ... Yes (1.22)
Checking if you have ExtUtils::MakeMaker 0 ... Yes (7.24)
Checking if you have URI 0 ... Yes (1.71)
Checking if you have vars 0 ... Yes (1.03)
Building and testing Net-HTTP-6.14
cp lib/Net/HTTP/NB.pm blib\lib\Net\HTTP\NB.pm
cp lib/Net/HTTPS.pm blib\lib\Net\HTTPS.pm
cp lib/Net/HTTP.pm blib\lib\Net\HTTP.pm
cp lib/Net/HTTP/Methods.pm blib\lib\Net\HTTP\Methods.pm
"C:\Strawberry\perl\bin\perl.exe" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib\lib', 'blib\arch')" t/*.t
# 
# Versions for all modules listed in MYMETA.json (including optional ones):
# 
# === Configure Requires ===
# 
#     Module              Want Have
#     ------------------- ---- ----
#     ExtUtils::MakeMaker  any 7.24
# 
# === Configure Suggests ===
# 
#     Module      Want    Have
#     -------- ------- -------
#     JSON::PP 2.27300 2.27400
# 
# === Build Requires ===
# 
#     Module              Want Have
#     ------------------- ---- ----
#     ExtUtils::MakeMaker  any 7.24
# 
# === Test Requires ===
# 
#     Module              Want     Have
#     ------------------- ---- --------
#     Data::Dumper         any    2.161
#     ExtUtils::MakeMaker  any     7.24
#     File::Spec           any  3.63_01
#     IO::Select           any     1.22
#     Socket               any    2.024
#     Test::More           any 1.302083
# 
# === Test Recommends ===
# 
#     Module         Want     Have
#     ---------- -------- --------
#     CPAN::Meta 2.120900 2.150010
# 
# === Runtime Requires ===
# 
#     Module                 Want  Have
#     ---------------------- ---- -----
#     Carp                    any  1.40
#     Compress::Raw::Zlib     any 2.071
#     IO::Socket::INET        any  1.35
#     IO::Uncompress::Gunzip  any 2.070
#     URI                     any  1.71
#     base                    any  2.23
#     strict                  any  1.11
#     vars                    any  1.03
#     warnings                any  1.36
# 
# === Runtime Suggests ===
# 
#     Module            Want  Have
#     ----------------- ---- -----
#     IO::Socket         any  1.38
#     IO::Socket::INET6  any  2.72
#     IO::Socket::IP     any  0.38
#     IO::Socket::SSL   1.38 2.043
#     Symbol             any  1.07
# 
t/00-report-prereqs.t .. ok
t/http-nb.t ............ skipped: This test doesn't work on Windows
t/http.t ............... ok
Bad file descriptor at t/live-https.t line 34.
# Looks like your test exited with 9 before it could output anything.
t/live-https.t ......... 
Dubious, test returned 9 (wstat 2304, 0x900)
Failed 6/6 subtests 
Server closed connection without sending any data back at C:\Users\rpeter\.cpanm\work\1493136176.12116\Net-HTTP-6.14\blib\lib/Net/HTTP/Methods.pm line 391.
# Looks like your test exited with 255 just after 3.
t/live.t ............... 
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 3/6 subtests 

#   Failed test 'base response length is less than 1024: 11847'
#   at t/rt-112313.t line 84.
Negative repeat count does nothing at t/rt-112313.t line 50.
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
Negative repeat count does nothing at t/rt-112313.t line 50.

#   Failed test 'reponse length increased by 1'
#   at t/rt-112313.t line 104.
#          got: '0'
#     expected: '1'
# Looks like you failed 19 tests of 40.
t/rt-112313.t .......... 
Dubious, test returned 19 (wstat 4864, 0x1300)
Failed 19/40 subtests 

Test Summary Report
-------------------
t/live-https.t       (Wstat: 2304 Tests: 0 Failed: 0)
  Non-zero exit status: 9
  Parse errors: Bad plan.  You planned 6 tests but ran 0.
t/live.t             (Wstat: 65280 Tests: 3 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 6 tests but ran 3.
t/rt-112313.t        (Wstat: 4864 Tests: 40 Failed: 19)
  Failed tests:  1, 4, 6, 8, 10, 12, 14, 17, 19, 21, 23
                25, 27, 30, 32, 34, 36, 38, 40
  Non-zero exit status: 19
Files=6, Tests=81, 26 wallclock secs ( 0.02 usr +  0.02 sys =  0.03 CPU)
Result: FAIL
Failed 3/6 test programs. 19/81 subtests failed.
dmake.exe:  Error code 147, while making 'test_dynamic'
-> FAIL Installing Net::HTTP failed. See C:\Users\rpeter\.cpanm\work\1493136176.12116\build.log for details. Retry with --force to force install it.

[genio]

Thanks for the report. I'll do some testing when I get home with access to a Windows machine.

[oalders]

@skaji possibly related to your changes?

[skaji]

First I want to make sure whether tests pass or not with Net::HTTP@6.13.

@pwr22 Could you test this please?

git clone https://github.com/libwww-perl/Net-HTTP
cd Net-HTTP
git checkout v6.13
touch LIVE_TESTS
perl Makefile.PL
make
make test

[update]

• If you don't have touch command, replace it by perl -E 'say 1' > LIVE_TESTS.
• If you don't have make command but dmake, then replace make by dmake.

[genio]

Or, cpanm --test-only Net::HTTP@6.13

[skaji]

We should create LIVE_TESTS file manually to run tests.
So cpanm --test-only Net::HTTP@6.13 does not run tests :)

[genio]

I see. Sorry, I completely missed that point.

[skaji]

As far as I know, Net::HTTP itself does not respect https?_proxy environment variable (LWP does).

@pwr22
I think you cannot access www.cpan.org or httpbin.org without proxy, so three tests (t/live-https.t, t/live.t, t/rt-112313.t) fail.

[pwr22]

@skaji enabling the live tests on 6.13 causes failures also. The difference being that they aren't run by default when installing that version though

I don't think it's a good idea to run live tests by default on build unless you have some way of doing a sanity check on connectivity first. Given that this is a HTTP module itself.... that's a bit of a bootstrapping problem

At work I'm stuck behind a firewall that stops pretty much all external connectivity. I'm 95% sure that's the problem here too

[pwr22]

Double checked on Test::RequiresInternet. Looks like it has no non-core deps so maybe the live tests could make use of it?

[skaji]

@pwr22 Thank you for checking.

@oalders @genio
I think we have two options to fix tests:

• (A) Use Test::RequiresInternet to check test network connectivity.
• (B) Use an opts-in environment variable (such as TEST_LIVE or LIVE_TEST). If it is true, then run tests. Otherwise, skip them.

[oalders]

I'd be inclined to try (A) and see how that works out.

[pwr22]

@oalders @skaji I'm happy to put together a PR for (A) if you'd like?

[skaji]

@pwr22 Yes! thanks!

[pwr22]

Hmm, I don't think (A) will be a viable fix for my environment. The firewall seems to respond in such a way that both the current tests sanity check (which I didn't know existed - sorry for missing that!) and Test::RequiresInternet don't catch it

[skaji]

@pwr22 Could you show us the outputs for the following commands?

perl -E "use Test::RequiresInternet q(www.cpan.org) =>  80; say q(pass)"
perl -E "use Test::RequiresInternet q(www.cpan.org) => 443; say q(pass)"
perl -E "use Test::RequiresInternet q(httpbin.org)  =>  80; say q(pass)"
perl -MLWP::UserAgent -E "print LWP::UserAgent->new(env_proxy => 0)->get(q(http://www.cpan.org))->dump"
perl -MLWP::UserAgent -E "print LWP::UserAgent->new(env_proxy => 1)->get(q(http://www.cpan.org))->dump"

[genio]

It seems more like this stems from PR20 and won't change with Test::RequiresInternet.

The fix would be to set $ENV{NO_NETWORK_TESTING}.

[genio]

In PowerShell:

$env:NO_NETWORK_TESTING = 1
cpanm Net::HTTP

[pwr22]

@skaji As you can see, 'non-authorised' network traffic returns something that my browser is happy to display but it upsets the live test

C:\Users\rpeter>perl -E "use Test::RequiresInternet q(www.cpan.org) =>  80; say q(pass)"
pass

C:\Users\rpeter>perl -E "use Test::RequiresInternet q(www.cpan.org) => 443; say q(pass)"
pass

C:\Users\rpeter>perl -E "use Test::RequiresInternet q(httpbin.org)  =>  80; say q(pass)"
pass

C:\Users\rpeter>perl -MLWP::UserAgent -E "print LWP::UserAgent->new(env_proxy => 0)->get(q(http://www.cpan.org))->dump"
HTTP/1.1 200 OK
Connection: Close
Content-Length: 11744
Content-Type: text/html; charset="utf-8"
Client-Date: Wed, 26 Apr 2017 07:31:49 GMT
Client-Peer: 151.101.50.49:80
Client-Response-Num: 1
Title: Hewlett Packard Enterprise Wrong Autocache
X-Meta-Charset: utf-8

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html lang="en"><head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<meta charset="utf-8">
<style>
body {
\tbackground-color:#425563;
}
h1   {
\tmargin: auto;
\tcolor:white;
\twidth: 60%;
\tfont-size:18px;
\tfont-family: 'Metric Semibold',Arial,sans-serif;
}
p    {
\tmargin: auto;
\tcolor:white;
\twidth: 60%;
\tfont-size:16px;
\tfont-family: 'Metric Light',Arial,sans-serif;
}
img    {
\tmargin: auto;
}
stro...
(+ 11232 more bytes not shown)

C:\Users\rpeter>perl -MLWP::UserAgent -E "print LWP::UserAgent->new(env_proxy => 1)->get(q(http://www.cpan.org))->dump"
HTTP/1.1 200 OK
Cache-Control: public, max-age=900, stale-while-revalidate=90, stale-if-error=172800
Connection: close
Date: Wed, 26 Apr 2017 07:32:07 GMT
Via: 1.1 varnish
Via: 1.1 varnish
Accept-Ranges: bytes
Age: 753
ETag: "179348-2132-54e0ca41580c0"
Server: Apache/2.2.15 (Red Hat)
Vary: Accept-Encoding
Content-Length: 8498
Content-Type: text/html
Last-Modified: Wed, 26 Apr 2017 07:18:03 GMT
Client-Date: Wed, 26 Apr 2017 07:32:07 GMT
Client-Peer: 16.46.41.11:8080
Client-Response-Num: 1
Fastly-Debug-Digest: 77034b31ca381bd3d6384e0f0bd19a20bac36d59f46306ebda1035dc65d1f57d
Link: <mailto:cpan+linkrelauthor@perl.org>; rel="author"
Link: <http://www.cpan.org/index.html>; rel="canonical"
Link: <misc/css/cpan.css>; rel="stylesheet"; type="text/css"
Title: The Comprehensive Perl Archive Network - www.cpan.org
X-Cache: HIT, HIT
X-Cache-Hits: 4, 1
X-Served-By: cache-lax8634-LAX, cache-lhr6325-LHR
X-Timer: S1493191928.754597,VS0,VE0

\n
<!DOCTYPE html>
<html lang="en">
<head>
    <title>The Comprehensive Perl Archive Network - www.cpan.org</title>
    <meta http-equiv="Content-Type" content="text/html;charset=utf-8" />    <link rel="author" href="mailto:cpan+linkrelauthor@perl.org" />
\t<link rel="canonical" href="http://www.cpan.org/index.html" />
\t<link type="text/css" rel="stylesheet" href="misc/css/cpan.css" />
\t
<!-- Copyright Jarkko Hietaniemi <jhi@iki.fi>
     1998,1999,2000,2001,2003,2003,2004,2005,2008
     Copyright 2010-2013 As...
(+ 7986 more bytes not shown)

@genio Thanks, is this a standard? I don't see it mentioned anywhere in the Net::HTTP docs

My general opinion is that this kind of testing is often brittle, it assumes that everyone has an open environment which sadly isn't true in the corporate world. IMO this testing should be opt-in, not out. Alternatively, since Net::HTTP doesn't honour the proxy variables, maybe if these are set this should also disable the live tests?

[genio]

@pwr22 Yes, it became a standard of sorts in 2014. If you like, you can follow through the logic in the Mailing List thread here.

Basically, there were too many dists trying to detect users' wishes on their own and they were all different. Test::RequiresInternet honors this environment variable, and our tests here mimic Test::RequiresInternet. The only difference in our test here is that we're not loading up the whole testing harness and whatnot first. This means we can fail those tests somewhat more quickly (nearly immediately).

All that being said, I do agree that maybe we should provide this information somewhere in the docs so others don't fall into this same problem. I think we'd be open to suggestion on how to word such documentation.