RHEL 6.x to RHEL 8.x migration - rrdgraph no longer works from NGINX

[rhelsysadmin]

This is a request for assistance, and we would likely be interested in a consulting engagement if warranted.

I am a RHEL sysadmin at a utilities company and have been asked to migrate NGINX from RHEL6 to RHEL8.

The solution on the RHEL6 host was put together by an outside contractor who is no longer available and the use of downloaded source complied locally was the approach.

My aim is to use only packages from the RHEL repository and only deviate from this minimally so that regular patching remains possible with only minimal recompilation from source being required.

After some work installing packages from the internal satellite server and meeting perl module requirements the solution almost works on RHEL 8. The gap is that the graphing fails with the following error:

Error while graphing "/opt/mrtg/central/FILE_NAME.png": cannot compile regular expression: Error while compiling regular expression ^(?:[^%]+|%%)%[-+ 0#]?[0-9](?:[.][0-9]+)?leEfFgG(?:%s)?(?:[^%]+|%%)$ at char 86: failed to get memory (^(?:[^%]+|%%)%[-+ 0#]?[0-9](?:[.][0-9]+)?leEfFgG(?:%s)?(?:[^%]+|%%)$)
Failed to open file "/opt/mrtg/central/FILE_NAME.png": No such file or directory

The only place I have been able to find that refers to the above message is here: https://trac.nginx.org/nginx/ticket/1403

The above nginx ticket leads me to this github page where I am hoping to find some useful guidance

The bit of customized perl module code that seems to be causing the error is comes from is this:

RRDs::graph @rrdArray;
my $ERR=RRDs::error;
warn "Error while graphing "$file-$period-$time.png": $ERR\n" if $ERR;
open( FILE, "$file-$period-$time.png" ) or warn "Failed to open file "$file-$period-$time.png": $!\n\n";
binmode(FILE);
my ($buf, $data, $n);
while (($n = read FILE, $data, 4) != 0)
{
$buf .= $data;
}
close( FILE );
unlink( "$file-$period-$time.png" ) or warn "Failed to clean up temporary file "$file-$period-$time.png": $!\n\n";
return $buf;

Not being a developer, I am admittedly out of my depth here and would greatly appreciate I nudge in the right direction (let me know if there is any more targeted information that I can provide) or if you are a developer and believe that you could work with me on a consulting basis to get this over the line I'd be like to hear from you.

Thanks if you have read this far and I apologize if I have posted this in the wrong place.

[oetiker]

the problem most likely is within the content of @rrdArray, I would add the following

use Data::Dumper;
warn Dumper \@rrdArray;

To see the content of the rrdArray.

[rhelsysadmin]

Thanks for the reply and the suggestion.

I have added the suggested lines to the failing .pm file and restarted NGINX.

Unfortunately, the only additional information in the error.log is "$VAR1 = [];"

So now the error.log gets this:
$VAR1 = [];
Error while graphing "/opt/mrtg/central/FILE_NAME.png": cannot compile regular expression: Error while compiling regular expression ^(?:[^%]+|%%)%[-+ 0#]?[0-9](?:[.][0-9]+)?leEfFgG(?:%s)?(?:[^%]+|%%)$ at char 86: failed to get memory (^(?:[^%]+|%%)%[-+ 0#]?[0-9](?:[.][0-9]+)?leEfFgG(?:%s)?(?:[^%]+|%%)$)
Failed to open file "/opt/mrtg/central/FILE_NAME.png": No such file or directory

Is there anything else I can do to try to get more meaningful debugging information?

Thanks in advance.

[rhelsysadmin]

Thanks again for the original reply, I was just hoping there might be something more that I can do to debug and isolate the issue.

Any thoughts?

[rhelsysadmin]

bump... still hoping for some guidance and still prepared to take a consulting offer to my manager for paid assistance.

Thanks.

[oetiker]

the regular expression compile problem could be related to an old pcre version or some thing like this

[rhelsysadmin]

The problem appears, to me, to be in the subroutine show below that defines "@rrdArray" with "my @rrdArray = ();" and then later fails to "RRDs::graph @rrdArray".

Looking again at the link: https://trac.nginx.org/nginx/ticket/1403

It seems that the subroutine should be calling "ngx_regex_compile()" instead of using PCRE which is no longer supported by NGINX.

I have looked around trying to see if I can easily get the subroutine to call "ngx_regex_compile()" but the information eludes me.

Would a simple edit make this happen? Or does the subroutine need to be completely rewritten?

If you could have one last look and advise I would greatly appreciate it.

Also, if I am in the wrong place, a pointer in the right direction would also be very much appreciated.

Thanks

#################### Subroutine Begin ###########################
sub main_screen_show_graph_img
{
my ( $self, $nodeId, $reference, $what, $period, $hidespikes ) = @;
my $sth = $self -> { dbh } -> prepare( "SELECT node,server FROM nodes,towns WHERE nodes.id=? and town=name" );
$sth -> execute( $nodeId );
my ( $node, $server ) = $sth -> fetchrow_array;
$sth -> finish;
my $file = "$path/$server/" . $nodeId . "" . lc($reference) . "_" . lc($what);
my @rrdArray = ();
warn Dumper @rrdArray;
my $time = time;
push( @rrdArray, "$file-$period-$time.png" );
if ( $hidespikes )
{
push( @rrdArray, "--logarithmic" );
push( @rrdArray, "--units" );
push( @rrdArray, "si" );
}
push( @rrdArray, "--x-grid" );

    my $start = "end-";
    my $periodSummary;
    my $step;
    if ( $period eq 'day' )
    {
            $periodSummary = "Day Summary";
            $start .= "32h";
            push( @rrdArray, "HOUR:1:DAY:1:HOUR:1:0:\%H" );
            $step = 300;
    }
    elsif ( $period eq 'week' )
    {
            $periodSummary = "Week Summary";
            $start .= "9d";
            push( @rrdArray, "DAY:1:WEEK:1:DAY:1:86400:\%a" );
            $step = 900;
    }
    elsif ( $period eq 'month' )
    {
            $periodSummary = "Month Summary";
            $start .= "5w";
            push( @rrdArray, "WEEK:1:MONTH:1:WEEK:1:604800:Week \%V" );
            $step = 3600;
    }
    else
    {
            $periodSummary = "Year Summary";
            $start .= "60w";
            push( @rrdArray, "WEEK:1:MONTH:1:MONTH:1:2592000:\%b" );
            $step = 43200;
    }
    push( @rrdArray, "--start" );
    push( @rrdArray, $start );
    push( @rrdArray, "--end" );
    push( @rrdArray, "now - 300s" );
    push( @rrdArray, "--step" );
    push( @rrdArray, $step );
    push( @rrdArray, "--imgformat" );
    push( @rrdArray, "PNG" );
    push( @rrdArray, "--width" );
    push( @rrdArray, "768" );
    push( @rrdArray, "--height" );
    push( @rrdArray, "200" );
    push( @rrdArray, "--slope-mode" );

    my $title = $node . " ";
    my $verticalLabel;
    push( @rrdArray, "DEF:ds0avg=" . $file . ".rrd:ds0:AVERAGE" );
    push( @rrdArray, "DEF:ds0max=" . $file . ".rrd:ds0:MAX" );
    push( @rrdArray, "DEF:ds1avg=" . $file . ".rrd:ds1:AVERAGE" );
    push( @rrdArray, "DEF:ds1max=" . $file . ".rrd:ds1:MAX" );
    my $postDS0;
    my $postDS1;
    my $postDS2;
    my $postDS3;
    my $label0;
    my $label1;
    my $label2;
    my $label3;
    my $short0;
    my $short1;
    if ( ( $reference eq 'memory' ) || ( $reference eq 'cpu' ) )
    {
            if ( $reference eq 'memory' )
            {
                    $title .= "Memory ";
                    $label0 = "Memory ";
                    $label1 = "IO     ";
                    $label2 = "Maximal 5 Minute Memory Utilisation";
                    $label3 = "Maximal 5 Minute IO";
            }
            else
            {
                    $title .= "CPU";
                    $label0 = "5m \%   ";
                    $label1 = "1m \%   ";
                    $label2 = "Maximal 5 Minute 5m CPU Utilisation";
                    $label3 = "Maximal 5 Minute 1m CPU Utilisation";
            }
            $title .= " Utilisation";
            push( @rrdArray, "--lower-limit" );
            if ( $hidespikes )
            {
                    push( @rrdArray, "1" );
            }
            else
            {
                    push( @rrdArray, "0" );
            }
            push( @rrdArray, "--upper-limit" );
            push( @rrdArray, "100" );
            push( @rrdArray, "--rigid" );
            $verticalLabel = "Utilisation %";
            $postDS0 = "ds0avg";
            $postDS1 = "ds1avg";
            $postDS2 = "ds0max";
            $postDS3 = "ds1max";
    }
    else
    {
            $title .= "Interface " . $reference . " ";
            $title .= "In/Out";
            if ( $what eq 'InOutErrors' )
            {
                    $title .= " Errors";
                    $verticalLabel = "Errors per second";
                    $label0 = "In  Eps";
                    $label1 = "Out Eps";
                    $label2 = "Maximal 5 Minute Incoming Errors Per Second";
                    $label3 = "Maximal 5 Minute Outgoing Errors Per Second";
            }
            else
            {
                    $verticalLabel = "Bits per second";
                    $label0 = "In  bps";
                    $label1 = "Out bps";
                    $label2 = "Maximal 5 Minute Incoming Bits Per Second";
                    $label3 = "Maximal 5 Minute Outgoing Bits Per Second";
            }
            push( @rrdArray, "CDEF:ds0bitsavg=ds0avg,8,*" );
            push( @rrdArray, "CDEF:ds1bitsavg=ds1avg,8,*" );
            push( @rrdArray, "CDEF:ds0bitsmax=ds0max,8,*" );
            push( @rrdArray, "CDEF:ds1bitsmax=ds1max,8,*" );
            $postDS0 = "ds0bitsavg";
            $postDS1 = "ds1bitsavg";
            $postDS2 = "ds0bitsmax";
            $postDS3 = "ds1bitsmax";
            $short0 = "\%s";
            $short1 = "\%s";
    }
    push( @rrdArray, "VDEF:vds0max=$postDS2,MAXIMUM" );
    push( @rrdArray, "VDEF:vds1max=$postDS3,MAXIMUM" );
    push( @rrdArray, "VDEF:vds0min=$postDS0,MINIMUM" );
    push( @rrdArray, "VDEF:vds1min=$postDS1,MINIMUM" );
    push( @rrdArray, "VDEF:vds0avg=$postDS0,AVERAGE" );
    push( @rrdArray, "VDEF:vds1avg=$postDS1,AVERAGE" );
    push( @rrdArray, "VDEF:vds0last=$postDS0,LAST" );
    push( @rrdArray, "VDEF:vds1last=$postDS1,LAST" );
    push( @rrdArray, "COMMENT:          " );
    push( @rrdArray, "COMMENT:    Maximum" );
    push( @rrdArray, "COMMENT:    Minimum" );
    push( @rrdArray, "COMMENT:    Average" );
    push( @rrdArray, "COMMENT:    Current\\n" );
    push( @rrdArray, "AREA:$postDS0#00ff00:$label0" );
    push( @rrdArray, "GPRINT:vds0max:\%10.2lf$short0" );
    push( @rrdArray, "GPRINT:vds0min:\%10.2lf$short0" );
    push( @rrdArray, "GPRINT:vds0avg:\%10.2lf$short0" );
    push( @rrdArray, "GPRINT:vds0last:\%10.2lf$short0\\n" );
    push( @rrdArray, "LINE1:$postDS1#0000ff:$label1" );
    push( @rrdArray, "GPRINT:vds1max:\%10.2lf$short1" );
    push( @rrdArray, "GPRINT:vds1min:\%10.2lf$short1" );
    push( @rrdArray, "GPRINT:vds1avg:\%10.2lf$short1" );
    push( @rrdArray, "GPRINT:vds1last:\%10.2lf$short1\\n" );
    if ( $period ne 'day' )
    {
            push( @rrdArray, "LINE1:$postDS2#006400:$label2" );
            push( @rrdArray, "LINE1:$postDS3#ff00ff:$label3" );
    }
    $title .= " $periodSummary";
    push( @rrdArray, "--title" );
    push( @rrdArray, $title );
    push( @rrdArray, "--vertical-label" );
    push( @rrdArray, $verticalLabel );
    push( @rrdArray, "--font" );
    push( @rrdArray, "DEFAULT:0:/usr/share/fonts/bitstream-vera/VeraMono.ttf" );

    RRDs::graph @rrdArray;
    my $ERR=RRDs::error;
    warn "Error while graphing \"$file-$period-$time.png\": $ERR\n" if $ERR;
    open( FILE, "$file-$period-$time.png" ) or warn "Failed to open file \"$file-$period-$time.png\": $!\n\n";
    binmode(FILE);
    my ($buf, $data, $n);
    while (($n = read FILE, $data, 4) != 0)
    {
            $buf .= $data;
    }
    close( FILE );
    unlink( "$file-$period-$time.png" ) or warn "Failed to clean up temporary file \"$file-$period-$time.png\": $!\n\n";
    return $buf;

}
#################### Subroutine End ###########################