Issues downloading Metagenomics data

Data resource IBDMDB
1

Hello,

I’m having issues downloading the metagenomics data as when I select all raw files, they’re too large to download. Is there another way of downloading the data/ all raw files, please?

Thank you very much for your help in advance!

Best wishes,
Hannah

2

Can you clarify where you’re trying to download from (URL)? And which files? That will help us to diagnose your issue.

3

Hello,

Thank you so much for your reply! I’m trying to download all fastq Metagenomics files from Metagenomes Raw Files | IBDMDB , but when I select all they are too large.

Thank you very much for your help!

Best wishes,
Hannah

4

Hello,

Thank you so much for your reply! I’m trying to download all fastq Metagenomics files from Metagenomes Raw Files | IBDMDB , but when I select all they are too large.

Thank you very much for your help!

Best wishes,
Hannah

5

Hi Franzosa,

I hope you’re well and thank you for your help with this- are you able to assist with downloading all data files, please?

Thank you very much!

Best wishes,
Hannah

6

Apologies, I’ve just realised that my messages containing the URL were flagged by the community so you were unable to see them - they are the Metagenomics fastq files, if that’s any help?

Many thanks,
Hannah

7

Hello!

Would you happen to have a path to the files/database so I can download them in terminal?

Thank you so much for your help!

Best wishes,
Hannah

8

It looks like you can get the URLs if you hover over them, e.g.:

https://ibdmdb.org/tunnel/static/HMP2_Pilot/WGS/1644/CSM5FZ3N_P.fastq.gz

Since the bulk download isn’t working, perhaps you could loop over the URLs in the terminal with e.g. wget and download them that way?

9

Hello @HannahFuller ,
Thank you for reaching out to the biobakery lab and apologies that you are facing issues for the bulk downloads.

We are currently in the verge of migrating the ibdmdb.org site to a new machine in which FTP will be enabled. This will resolve the issue of bulk download for the large data files. The roll over date is 2023 July 1st. We will update you in the same thread as soon as the FTP is enabled for the site.

Regards,
Sagun

10

That’s great, thank you - I’ll try that!

11

That’s great, thank you so much!!

Best wishes,
Hannah

12

Hello! I hope you’re well. The site is back up but the bulk download isn’t working (as the request is too large). Is there a way to get a list of the files so I can download them in terminal using wget?

Thank you for your help

Best wishes,
Hannah

13

Hello Sagun,

Thank you for your help with these queries- could I please ask for the link to the FTP? I can see the address and password, however when I try and search the address it comes up with an error.

Thank you!

Best wishes,
Hannah