How to use Linux hw-probe behind a HTTP Proxy (for probe upload to linux-hardware.org)

Written by Claudio Kuenzler - 0 comments

Published on June 11th 2021 - last updated on June 15th 2021 - Listed in Linux Perl Coding Hardware

---

Behind the public hardware database linux-hardware.org is basically one open source program: hw-probe. This little helper tool collects information from local hardware devices, tries to determine whether or not they are correctly working and uploads the "probe" into the database of linux-hardware.org.

This database is a great helper if you want to buy new hardware but Linux compatibility is not (well) documented of that hardware device. I've used this database to check Linux compatibility in Spring 2021 when my main workstation for daily work received a hardware upgrade (ASRock B550 and AMD Ryzen 5 3600 on Linux Mint 20.1 with Kernel 5.8).

Probe upload fails

To upload the probe, an Internet connection is obviously required. However there are many environments, where direct Internet access is not possible or blocked by a firewall. hw-probe then fails to upload the probe:

root@linux:~# sudo -E hw-probe -all -upload
Probe for hardware ... Ok
Reading logs ... WARNING: X11-related logs are not collected (try to run 'xhost +local:' to enable access or run as root by su)
Ok
read failed: Connection reset by peer at /usr/share/perl5/LWP/Protocol/http.pm line 382.

ERROR: failed to upload data

In such environments, an outgoing http/https proxy is often used. But even by defining the proxy using environment variables, the probe upload would still fail:

root@linux:~# export https_proxy=http://myproxy.example.com:3128
root@linux:~# export http_proxy=http://myproxy.example.com:3128
root@linux:~# sudo -E hw-probe -all -upload
Probe for hardware ... Ok
Reading logs ... WARNING: X11-related logs are not collected (try to run 'xhost +local:' to enable access or run as root by su)
Ok
read failed: Connection reset by peer at /usr/share/perl5/LWP/Protocol/http.pm line 382.

ERROR: failed to upload data

The errors reveal that in the background the Perl module LWP is used.

Analyzing the source code

As /usr/bin/hw-probe is a Perl script, the source code can be read and modified (without having to run a ton of compilers). The relevant upload part indeed starts a new LWP::UserAgent connection in the script's postRequest function:

sub postRequest($$$)
{
    my ($UploadURL, $Data, $SSL) = @_;

    require LWP::UserAgent;

    my $UAgent = LWP::UserAgent->new(parse_head => 0);

    if($SSL eq "NoSSL" or not checkModule("Mozilla/CA.pm"))
    {
        $UploadURL=~s/\Ahttps:/http:/g;
        $UAgent->ssl_opts(verify_hostname => 0, SSL_verify_mode => 0x00);
    }

    my $Res = $UAgent->post(
        $UploadURL,
        Content_Type => "form-data",
        Content => $Data
    );

    my $Out = $Res->{"_content"};

    if(not $Out) {
        return $Res->{"_headers"}{"x-died"};
    }

    return $Out;
}

The LWP::UserAgent module also supports a proxy as option, but obviously this is not implemented here.

Adding a proxy option to hw-probe

To add support for outgoing http/https proxies, I modified the postRequest function, checking whether or not the command line parameter -proxy (which is new) was used and then use the UserAgent option 'proxy':

sub postRequest($$$)
{
    my ($UploadURL, $Data, $SSL) = @_;

    require LWP::UserAgent;

    my $UAgent = LWP::UserAgent->new(parse_head => 0);

    if($Opt{"Proxy"}) {
        my $proxy = $Opt{"Proxy"};
        $UAgent->proxy([ [ 'http', 'https' ] => "http://$proxy" ]);
    }

    if($SSL eq "NoSSL" or not checkModule("Mozilla/CA.pm"))
    {
        $UploadURL=~s/\Ahttps:/http:/g;
        $UAgent->ssl_opts(verify_hostname => 0, SSL_verify_mode => 0x00);
    }

    my $Res = $UAgent->post(
        $UploadURL,
        Content_Type => "form-data",
        Content => $Data
    );

    my $Out = $Res->{"_content"};

    if(not $Out) {
        return $Res->{"_headers"}{"x-died"};
    }

    return $Out;
}

To see the full diff:

$ git diff
diff --git a/hw-probe.pl b/hw-probe.pl
index 3d1a084..8a2b0a2 100644
--- a/hw-probe.pl
+++ b/hw-probe.pl
@@ -167,6 +167,7 @@ GetOptions("h|help!" => \$Opt{"Help"},
   "upload|confirm-upload-of-hashed-ids!" => \$Opt{"Upload"},
   "hwinfo-path=s" => \$Opt{"HWInfoPath"},
   "log!" => \$Opt{"ShowLog"},
+  "proxy=s" => \$Opt{"Proxy"},
 # Inventory
   "inventory|inventory-id|i|group|g=s" => \$Opt{"Group"},
   "generate-inventory|generate-inventory-id|get-inventory-id|get-group!" => \$Opt{"GenerateGroup"},
@@ -355,6 +356,9 @@ GENERAL OPTIONS:
   -hwinfo-path PATH
       Path to a local hwinfo binary.

+  -proxy address:port
+      Set outgoing http/https proxy using syntax: proxy.domain.local:3128
+
 INVENTORY OPTIONS:
   -i|-inventory-id ID
       Mark the probe by inventory ID.
@@ -2475,6 +2479,11 @@ sub postRequest($$$)
     require LWP::UserAgent;

     my $UAgent = LWP::UserAgent->new(parse_head => 0);
+
+    if($Opt{"Proxy"}) {
+        my $proxy = $Opt{"Proxy"};
+        $UAgent->proxy([ [ 'http', 'https' ] => "http://$proxy" ]);
+    }

     if($SSL eq "NoSSL" or not checkModule("Mozilla/CA.pm"))
     {

The changes were pushed to the upstream project as Pull Request #102.

Running hw-probe with a proxy

Running hw-probe now with the new -proxy parameter and the proxy defined in the command line:

root@linux:~# sudo -E hw-probe -all -upload -proxy myproxy.example.com:3128
Probe for hardware ... Ok
Reading logs ... WARNING: X11-related logs are not collected (try to run 'xhost +local:' to enable access or run as root by su)
Ok
Uploaded to DB, Thank you!

Probe URL: https://linux-hardware.org/?probe=a35e7d3eb0

And the probe was sucessfully uploaded!

Open Source Rulez!

Update: The code has been merged

The code to add the possibility of using an outgoing proxy has been merged into the upstream project by the hw-probe maintainer.

Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.