nginx-hmac-secure-link #18

nexusofdoom · 2018-09-18T10:23:56Z

revers proxy nginx with nginx-hmac-secure-link
can this be done?
nexusofdoom/lancache-installer#20

agile6v · 2018-09-19T13:23:41Z

Sorry. I don't understand your question.
Please explain in detail.

Thanks.

nexusofdoom · 2018-09-19T13:29:11Z

caching nginx box setup like so.. online host uses Prerequisites for Token Authentication from akimia it addess this to the request acl=/*~hmac=6a10b3f602ffde88c02cd1b89665bfdfdc0fc84c3cf7049752931ad732654fab

the client points to my nginx caching box and ask to get the file but it never gets a hit on the key.

is there a way to rewrite the new token to the end of the request ? or something more I can do?

this is the url it requests "akamai-gamecdn.blackdesertonline.com/live001/game/download/131/BlackDesertOnline885.bcab?begin=1238753107&end=1247683342&hdnts=st=1537229206exp=1537230106acl=/live001/game/download/*!/live001/game/patch/*~hmac=c2d249db5aff5d6940e03447517ba12b41fdb9aa204cd3c762a9265f445beade"

and below is what the key shows
that it miss
/live001/game/download/131/BlackDesertOnline885.bcab

agile6v · 2018-09-19T14:34:28Z

Hello

What is the response code for this request?

"akamai-gamecdn.blackdesertonline.com/live001/game/download/131/BlackDesertOnline885.bcab?begin=1238753107&end=1247683342&hdnts=st=1537229206exp=1537230106acl=/live001/game/download/*!/live001/game/patch/*~hmac=c2d249db5aff5d6940e03447517ba12b41fdb9aa204cd3c762a9265f445beade"

nexusofdoom · 2018-09-19T14:38:28Z

where do you want me to get the response code ?
I have httpry installed along with I can look at the nginx log or do you want me to run wireshark on the client pc?

nexusofdoom · 2018-09-19T14:41:48Z

note in httpry its code 200

agile6v · 2018-09-19T14:42:02Z

You can get it using curl. I want to determine if the request is returned 200.

nexusofdoom · 2018-09-19T14:51:31Z

Ok will do when I get back from work tonight. thanks.

agile6v · 2018-09-19T15:09:41Z

Sorry, I can only judge by my experience.
I think you should request the origin website, check if the response is allowed caching. And also send me a copy.

nexusofdoom · 2018-09-19T23:29:22Z

direct test talking to there server
curl "akamai-gamecdn.blackdesertonline.com/live001/game/download/131/BlackDesertOnline885.bcab?begin=1238753107&end=1247683342&hdnts=st=1537229206~~exp=1537230106~~acl=/live001/game/download/*!/live001/game/patch/*~hmac=
c2d249db5aff5d6940e03447517ba12b41fdb9aa204cd3c762a9265f445beade"
curl : An error occurred while processing your request.
Reference #199.5ef00a17.1537399717.1bba468b
At line:1 char:1

curl "akamai-gamecdn.blackdesertonline.com/live001/game/download/131/ ...

  + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
  + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

nexusofdoom · 2018-09-19T23:30:54Z

if I use lancache I get this, note it saves files to the cache and the client gets the file but the next time it downloads the acl= changes and it comes back as a miss in nginx and it re-downloads the file all over again.

curl "akamai-gamecdn.blackdesertonline.com/live001/game/download/131/BlackDesertOnline885.bcab?begin=1238753107&end=1247683342&hdnts=st=1537229206~~exp=1537230106~~acl=/live001/game/download/*!/live001/game/patch/*~hmac=
c2d249db5aff5d6940e03447517ba12b41fdb9aa204cd3c762a9265f445beade"

StatusCode : 200
StatusDescription : OK
Content : ��À(Y Ð Ü·�0 � 4� À" �3 �òR� � ô7 x/� �Â ��aq� � lg� @ç� }y ´ÞLÏ� � ¬N
P� N� º@ÔÂ üO
P� N� ù�m6
LQ
pC 0� 3ë+© � ¼�
�= x� ¨A�ß TÒ
�C ...
RawContent : HTTP/1.1 200 OK
Connection: keep-alive
X-Upstream-Cache-Status: HIT
Accept-Ranges: bytes
Content-Length: 8930236
Content-Type: text/plain
Date: Wed, 19 Sep 2018 23:25:32 GMT
ETag: "5518bf566eb7...
Forms : {}
Headers : {[Connection, keep-alive], [X-Upstream-Cache-Status, HIT], [Accept-Ranges, bytes], [Content-Length, 8930236]...}
Images : {}
InputFields : {}
Links : {}
ParsedHtml : mshtml.HTMLDocumentClass
RawContentLength : 8930236

agile6v · 2018-09-20T01:22:21Z

Hello

I noticed $request_url is used as key in your nginx conf.
$request_uri : full original request URI (with arguments)

nexusofdoom · 2018-09-20T01:29:35Z

this is the testing config I am playing with.
/etc/nginx/vhosts-enabled/lancache-pearlabyss.conf

server {
listen lancache-pearlabyss deferred default;
server_name pearlabyss _;
# DNS entries: lancache-
access_log /srv/lancache/logs/Access/pearlabyss.log main buffer=128k flush=1m;
access_log /srv/lancache/logs/Keys/pearlabyss.log keys_uri buffer=128k flush=1m;
error_log /srv/lancache/logs/Errors/pearlabyss.log;

# Default Node
include lancache/resolver;
#include lancache/cache-key-range;
    location / {
            set $no_cache 0;
            if ($request_uri ~* "(/live001/game/config/config.patch.version)") {
            set $no_cache 1;
            }
            proxy_no_cache $no_cache;
            proxy_cache_bypass $no_cache;
            proxy_bind 10.0.4.20;
            proxy_ignore_headers Expires Cache-Control;
            #proxy_set_header Range $http_range;
            #proxy_set_header If-Range $http_if_range;
            #proxy_cache_key "$server_name$uri";
            #proxy_cache_key "$scheme$request_method$host$uri$is_args$args";
            #proxy_cache_key "$uri$is_args$args$slice_range";
            #proxy_set_header X-Real-IP $remote_addr;
            #proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            #proxy_cache_valid 200  206 320d;
            #proxy_cache_key $scheme$host$proxy_host$request_uri;
            #proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            #proxy_set_header Host $host;
            #proxy_set_header X-Forwarded-Proto $scheme;
            proxy_read_timeout 150;
            # Use Origin cache
            proxy_cache pearlabyss;
            include lancache/proxy-cache;
            #include lancache/cache-range;
            include lancache/cache-key-default;
    }

}

agile6v · 2018-09-20T01:47:17Z

Your should use $uri instead of $request_uri as proxy_cache_key.

nexusofdoom · 2018-09-20T01:57:17Z

so change this
include lancache/cache-key-default;
}
}

to

        include lancache/cache-key-uri;
}

}

nexusofdoom · 2018-09-20T01:59:50Z

I did this

server {
listen lancache-pearlabyss deferred default;
server_name pearlabyss _;
# DNS entries: lancache-
access_log /srv/lancache/logs/Access/pearlabyss.log main buffer=128k flush=1m;
access_log /srv/lancache/logs/Keys/pearlabyss.log keys_uri buffer=128k flush=1m;
error_log /srv/lancache/logs/Errors/pearlabyss.log;

# Default Node
include lancache/resolver;
#include lancache/cache-key-range;
    location / {
            set $no_cache 0;
            if ($request_uri ~* "(/live001/game/config/config.patch.version)") {
            set $no_cache 1;
            }
            proxy_no_cache $no_cache;
            proxy_cache_bypass $no_cache;
            proxy_bind 10.0.4.20;
            proxy_ignore_headers Expires Cache-Control;
            #proxy_set_header Range $http_range;
            #proxy_set_header If-Range $http_if_range;
            proxy_cache_key "$uri";

nexusofdoom · 2018-09-20T02:00:51Z

once i did that this is the error I get in the client

nexusofdoom · 2018-09-20T02:02:48Z

httpry on the nginx server
2018-09-19 21:02:01 10.0.4.45 10.0.4.39 > GET akamai-gamecdn.blackdesertonline.com /live001/game/download/132/BlackDesertOnline891.bcab?begin=0&end=12&hdnts=st=1537408917~~exp=1537409817~~acl=/live001/game/download/*!/live001/game/patch/hmac=a1280702ad1721200608c2af28d41b3156e7c46208f4dc5c43f627e6cd5b773b HTTP/1.1 - -
2018-09-19 21:02:01 10.0.4.39 10.0.4.45 < - - - HTTP/1.1 200 OK
2018-09-19 21:02:01 10.0.4.45 10.0.4.39 > GET akamai-gamecdn.blackdesertonline.com /live001/game/download/132/BlackDesertOnline891.bcab?begin=13&end=359540&hdnts=st=1537408917exp=1537409817~acl=/live001/game/download/!/live001/game/patch/*~hmac=a1280702ad1721200608c2af28d41b3156e7c46208f4dc5c43f627e6cd5b773b HTTP/1.1 - -
2018-09-19 21:02:01 10.0.4.39 10.0.4.45 < - - - HTTP/1.1 200 OK
2

nexusofdoom · 2018-09-20T02:05:21Z

if i change the hmac= from 773b to 733a i get this message in chrome
An error occurred while processing your request.
Reference #199.5ef00a17.1537409068.1bff07c8

nexusofdoom · 2018-09-20T02:06:52Z

curl "akamai-gamecdn.blackdesertonline.com/live001/game/download/131/BlackDesertOnline885.bcab"
curl : An error occurred while processing your request.
Reference #199.3cb2ef50.1537409184.76a1387
At line:1 char:1

curl "akamai-gamecdn.blackdesertonline.com/live001/game/download/131/ ...

  + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebExc
 eption
  + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

agile6v · 2018-09-20T02:09:25Z

curl -v "akamai-gamecdn.blackdesertonline.com/live001/game/download/131/BlackDesertOnline885.bcab"

It will return 403. It means that the certification did not pass.

agile6v · 2018-09-20T02:12:48Z

I think you should be aware of what kind of request your application make to the nginx proxy, including the headers of each request. Then manually curl these request.

nexusofdoom · 2018-09-20T02:15:03Z

curl -v "akamai-gamecdn.blackdesertonline.com/live001/game/download/131/BlackDesertOnline885.bcab?beg
in=1238753107&end=1247683342&hdnts=st=1537229206~~exp=1537230106~~acl=/live001/game/download/*!/live001/game/patch/hmac=
b99eb03688087d7664b1e5cb69a56c57345dd928a4695dd18257486c2c94cd94"
VERBOSE: GET
http://akamai-gamecdn.blackdesertonline.com/live001/game/download/131/BlackDesertOnline885.bcab?begin=1238753107&end=12
47683342&hdnts=st=1537229206exp=1537230106~acl=/live001/game/download/!/live001/game/patch/*~hmac=b99eb03688087d7664b
1e5cb69a56c57345dd928a4695dd18257486c2c94cd94 with 0-byte payload
curl : An error occurred while processing your request.
Reference #199.45b2ef50.1537409669.788a8c0
At line:1 char:1

curl -v "akamai-gamecdn.blackdesertonline.com/live001/game/download/1 ...

  + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebExc
 eption
  + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

nexusofdoom · 2018-09-20T02:15:28Z

should I be running wireshark when the application start the request on the client pc?

agile6v · 2018-09-20T02:20:48Z

Yes. The tool you are good at can easily analyze the problem.

nexusofdoom · 2018-09-20T02:30:42Z

I am getting this now says hit but its not pulling the 8mb file

wget "akamai-gamecdn.blackdesertonline.com/live001/game/download/131/BlackDesertOnline885.bcab?begin=
1238753107&end=1247683342&hdnts=st=1537229206~~exp=1537230106~~acl=/live001/game/download/*!/live001/game/patch/"

StatusCode : 200
StatusDescription : OK
Content : bdoc�/|� ÐUFK
RawContent : HTTP/1.1 200 OK
Connection: keep-alive
X-N: S
X-Upstream-Cache-Status: HIT
Accept-Ranges: bytes
Content-Length: 13
Content-Type: text/plain
Date: Thu, 20 Sep 2018 02:30:17 GMT
ETag: "5518bf566...
Forms : {}
Headers : {[Connection, keep-alive], [X-N, S], [X-Upstream-Cache-Status, HIT], [Accept-Ranges, bytes]...}
Images : {}
InputFields : {}
Links : {}
ParsedHtml : mshtml.HTMLDocumentClass
RawContentLength : 13

agile6v · 2018-09-20T02:46:23Z

The cached content is incorrect. Delete it and request it again.

nexusofdoom · 2018-09-20T02:56:08Z

still no go after deleting cache i see it trying to fill the cache and this is one of the files in cache directory

^E^@^@^@^@^@^@^@��^Y^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@q^D�[^@^@^@^@�^K�[^@^@^@^@�ì�^@^@�^A�^B-"a95d9304b94861f6c3a8367fe93eac54:1537344593"^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@$
KEY: pearlabyss/live001/game/download/132/BlackDesertOnline891.bcab
HTTP/1.0 200 OK
Server: Apache
Accept-Ranges: bytes
Content-Type: text/plain
Last-Modified: Wed, 19 Sep 2018 08:09:53 GMT
ETag: "a95d9304b94861f6c3a8367fe93eac54:1537344593"
Date: Thu, 20 Sep 2018 02:53:08 GMT
Content-Length: 13
Connection: close
X-N: S

bdoc^Ah|^E^@�2)?

nexusofdoom · 2018-09-20T03:13:25Z

[19/Sep/2018:22:12:08 -0500] - REQUEST="pearlabyss/live001/game/download/132/BlackDesertOnline891.bcab?begin=109683834&end=109684249&hdnts=st=1537413088~~exp=1537413988~~acl=/live001/game/download/*!/live001/game/patch/*~hmac=a7abcd104bb93d2c54e918247c87f0abf8cd69866751f1a8ec35308834897906" - KEY="pearlabyss/live001/game/download/132/BlackDesertOnline891.bcab" - CACHE="MISS"

agile6v · 2018-09-20T03:27:59Z

Is this request MISS every time? Has it ever had HIT?

nexusofdoom · 2018-09-20T13:47:21Z

when I get it to HIT then the client software does not like it
maybe a rewrite inside of the config? since this changes every time
"*~hmac=a7abcd104bb93d2c54e918247c87f0abf8cd69866751f1a8ec35308834897906"

also when it hit's it only downloads like a 1kb file when the file is 30gb.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nginx-hmac-secure-link #18

nginx-hmac-secure-link #18

nexusofdoom commented Sep 18, 2018

agile6v commented Sep 19, 2018

nexusofdoom commented Sep 19, 2018

agile6v commented Sep 19, 2018

nexusofdoom commented Sep 19, 2018

nexusofdoom commented Sep 19, 2018

agile6v commented Sep 19, 2018

nexusofdoom commented Sep 19, 2018

agile6v commented Sep 19, 2018

nexusofdoom commented Sep 19, 2018

nexusofdoom commented Sep 19, 2018

agile6v commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

agile6v commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

agile6v commented Sep 20, 2018

agile6v commented Sep 20, 2018 •

edited

Loading

nexusofdoom commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

agile6v commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

agile6v commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

agile6v commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018 •

edited

Loading

nginx-hmac-secure-link #18

nginx-hmac-secure-link #18

Comments

nexusofdoom commented Sep 18, 2018

agile6v commented Sep 19, 2018

nexusofdoom commented Sep 19, 2018

agile6v commented Sep 19, 2018

nexusofdoom commented Sep 19, 2018

nexusofdoom commented Sep 19, 2018

agile6v commented Sep 19, 2018

nexusofdoom commented Sep 19, 2018

agile6v commented Sep 19, 2018

nexusofdoom commented Sep 19, 2018

nexusofdoom commented Sep 19, 2018

agile6v commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

agile6v commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

agile6v commented Sep 20, 2018

agile6v commented Sep 20, 2018 • edited Loading

nexusofdoom commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

agile6v commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

agile6v commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018

agile6v commented Sep 20, 2018

nexusofdoom commented Sep 20, 2018 • edited Loading

agile6v commented Sep 20, 2018 •

edited

Loading

nexusofdoom commented Sep 20, 2018 •

edited

Loading