Skip to content

hg38_screen_v10_clust.regions_vs_motifs.rankings.feather (hg38 SCREEN v10 cisTarget rankings database) returns HTTP 404 #61

Open

Description

@yangxvlin

hg38 SCREEN v10 cisTarget rankings database returns HTTP 404

Questions

  1. Has the cisTarget resource server been migrated or reconfigured recently?
  2. Is webserver2.aertslab.org (134.58.50.9) still the correct backend for resources.aertslab.org?
  3. Could the Apache virtual-host or document-root configuration for /cistarget/ be broken?
  4. Is there an alternative mirror or download URL for the hg38 SCREEN v10 rankings database?
  5. Could you also fix the TLS certificate configuration for resources.aertslab.org?

I would appreciate confirmation of whether this is a known server-side issue and guidance on how to obtain the database.

Findings / Actual behavior

I'm trying to download the following precomputed cisTarget database:

Target file:

hg38_screen_v10_clust.regions_vs_motifs.rankings.feather (approximately 33 GB)

Official directory:

https://resources.aertslab.org/cistarget/databases/homo_sapiens/hg38/screen/mc_v10_clust/region_based/

The official directory listing still advertises the database and its checksum file. However, all download attempts from my environment fail.

The observed behavior:

  • DNS resolves resources.aertslab.org to webserver2.aertslab.org (134.58.50.9).
  • The TLS certificate presented by the server is valid for aertslab.org and www.aertslab.org, but not resources.aertslab.org.
  • HTTPS requests to the original resource hostname return HTTP 404 when certificate verification is bypassed.
  • HTTPS requests to the canonical hostname aertslab.org also return HTTP 404.
  • Requests to webserver2.aertslab.org return HTTP 404.
  • The /cistarget/ root, /cistarget/databases/, the target database directory, and /cistarget/motif_collections/ all return HTTP 404.
  • The .zsync metadata file also returns HTTP 404.
  • The official zsync_curl utility download returns an HTML 404 page instead of an executable.
  • A native Windows zsync-compatible client (gozsync.exe) reproduces the same failures.

This suggests a server-side resource hosting, Apache routing, or deployment issue rather than a problem with the download client.

Commands executed and results

Environment:

  • OS: Windows
  • Terminal: MinGW / Git Bash
  • Architecture: Windows x64
  • Download client: gozsync.exe
  • DNS resolver: 10.88.73.5

1. DNS resolution

Command:

nslookup resources.aertslab.org

Result:

Server:  UnKnown
Address:  10.88.73.5

Non-authoritative answer:
Name:    webserver2.aertslab.org
Address: 134.58.50.9
Aliases: resources.aertslab.org

2. TLS certificate inspection

Command:

openssl s_client \
  -connect resources.aertslab.org:443 \
  -servername resources.aertslab.org \
  </dev/null 2>/dev/null \
  | openssl x509 -noout -subject -issuer -ext subjectAltName

Result:

subject=CN = aertslab.org
issuer=C = US, O = Let's Encrypt, CN = YR2

X509v3 Subject Alternative Name:
    DNS:aertslab.org, DNS:www.aertslab.org

The certificate does not include resources.aertslab.org in its Subject Alternative Names.

3. Direct HTTPS request to the database

Command:

curl -I "https://resources.aertslab.org/cistarget/databases/homo_sapiens/hg38/screen/mc_v10_clust/region_based/hg38_screen_v10_clust.regions_vs_motifs.rankings.feather"

Result:

curl: (60) SSL: no alternative certificate subject name
matches target host name 'resources.aertslab.org'

4. Download with TLS verification disabled

Command:

curl -k -L -C - -O \
"https://resources.aertslab.org/cistarget/databases/homo_sapiens/hg38/screen/mc_v10_clust/region_based/hg38_screen_v10_clust.regions_vs_motifs.rankings.feather"

Result:

100   299  100   299

The downloaded file was only 299 bytes.

Inspecting the file:

cat hg38_screen_v10_clust.regions_vs_motifs.rankings.feather

Result:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL was not found on this server.</p>
<hr>
<address>Apache Server at aertslab.org Port 443</address>
</body></html>

5. HTTPS HEAD request with TLS verification disabled

Command:

curl -k -L -v -I \
"https://resources.aertslab.org/cistarget/databases/homo_sapiens/hg38/screen/mc_v10_clust/region_based/hg38_screen_v10_clust.regions_vs_motifs.rankings.feather"

Relevant result:

* Connected to resources.aertslab.org (134.58.50.9) port 443
* SSL connection using TLSv1.3
* Server certificate:
* subject: CN=aertslab.org

> HEAD /cistarget/databases/homo_sapiens/hg38/screen/mc_v10_clust/region_based/hg38_screen_v10_clust.regions_vs_motifs.rankings.feather HTTP/1.1
> Host: resources.aertslab.org

< HTTP/1.1 404 Not Found
< Server: Apache
< Content-Type: text/html; charset=iso-8859-1

6. Test canonical hostnames

Command 1:

curl -L -I \
"https://aertslab.org/cistarget/databases/homo_sapiens/hg38/screen/mc_v10_clust/region_based/hg38_screen_v10_clust.regions_vs_motifs.rankings.feather"

Result:

HTTP/1.1 404 Not Found
Server: Apache

Command 2:

curl -L -I \
"https://www.aertslab.org/cistarget/databases/homo_sapiens/hg38/screen/mc_v10_clust/region_based/hg38_screen_v10_clust.regions_vs_motifs.rankings.feather"

Result:

HTTP/1.1 404 Not Found
Server: Apache

7. Test the canonical server hostname directly

Command:

curl -k -I \
"https://webserver2.aertslab.org/cistarget/"

Result:

HTTP/1.1 404 Not Found
Server: Apache

8. Test HTTP instead of HTTPS

Command:

curl -I "http://resources.aertslab.org/cistarget/"

Result:

HTTP/1.1 302 Found
Location: https://aertslab.org/cistarget/

Following the redirect leads to the HTTPS endpoint that returns 404.

9. Test parent directories

Commands:

curl -k -I \
"https://resources.aertslab.org/cistarget/"

curl -k -I \
"https://resources.aertslab.org/cistarget/databases/"

curl -k -I \
"https://resources.aertslab.org/cistarget/databases/homo_sapiens/hg38/screen/mc_v10_clust/region_based/"

curl -k -I \
"https://resources.aertslab.org/cistarget/motif_collections/"

Result for all four requests:

HTTP/1.1 404 Not Found
Server: Apache

10. Test the .zsync metadata file

Command:

curl -k -L -I \
"https://resources.aertslab.org/cistarget/databases/homo_sapiens/hg38/screen/mc_v10_clust/region_based/hg38_screen_v10_clust.regions_vs_motifs.rankings.feather.zsync"

Result:

HTTP/1.1 404 Not Found
Server: Apache

11. Test alternative hostname

Command:

curl -k -I \
"https://www.resources.aertslab.org/cistarget/"

Result:

HTTP/1.1 404 Not Found
Server: Apache

12. Test DNS-independent IP routing

Command:

curl -k -I \
  --resolve resources.aertslab.org:443:134.58.50.9 \
  "https://resources.aertslab.org/cistarget/"

Result:

HTTP/1.1 404 Not Found
Server: Apache

13. Test the official zsync_curl utility download

Command:

curl -k -L -o zsync_curl \
"https://resources.aertslab.org/cistarget/zsync_curl"

Inspection:

file zsync_curl

Result:

zsync_curl: HTML document, ASCII text
head -5 zsync_curl

Result:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>

14. Test native Windows gozsync

I obtained a native Windows zsync-compatible client, gozsync.exe.

Command:

./gozsync.exe --help

The executable works and reports support for standard zsync control files, HTTP Range requests, and optional seed files.

Test 1: Original resource hostname

./gozsync.exe \
  -o hg38_screen_v10_clust.regions_vs_motifs.rankings.feather \
  "https://resources.aertslab.org/cistarget/databases/homo_sapiens/hg38/screen/mc_v10_clust/region_based/hg38_screen_v10_clust.regions_vs_motifs.rankings.feather.zsync"

Result:

gozsync: GET https://resources.aertslab.org/cistarget/databases/homo_sapiens/hg38/screen/mc_v10_clust/region_based/hg38_screen_v10_clust.regions_vs_motifs.rankings.feather.zsync:

tls: failed to verify certificate:
x509: certificate is valid for aertslab.org, www.aertslab.org,
not resources.aertslab.org

Test 2: Canonical hostname

./gozsync.exe \
  -o hg38_screen_v10_clust.regions_vs_motifs.rankings.feather \
  "https://aertslab.org/cistarget/databases/homo_sapiens/hg38/screen/mc_v10_clust/region_based/hg38_screen_v10_clust.regions_vs_motifs.rankings.feather.zsync"

Result:

gozsync: GET https://aertslab.org/cistarget/databases/homo_sapiens/hg38/screen/mc_v10_clust/region_based/hg38_screen_v10_clust.regions_vs_motifs.rankings.feather.zsync: 404 Not Found

References

Thank you for looking into this!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions