Hello,
Thanks for your great contributions for the advanced of bioinfo analysis.
i meet a terrible error when using metaphlan and have no idea to handle it after google and reinstall the software. please help me, the detail infos are as follows,
Traceback (most recent call last):
File “/public5/home/sch7108/.conda/envs/metagenome/bin/metaphlan”, line 10, in
sys.exit(main())
File “/public5/home/sch7108/.conda/envs/metagenome/lib/python3.9/site-packages/metaphlan/metaphlan.py”, line 1084, in main
mpa_pkl = pickle.load( a )
File “/public5/home/sch7108/.conda/envs/metagenome/lib/python3.9/bz2.py”, line 161, in peek
return self._buffer.peek(n)
File “/public5/home/sch7108/.conda/envs/metagenome/lib/python3.9/_compression.py”, line 68, in readinto
data = self.read(len(byte_view))
File “/public5/home/sch7108/.conda/envs/metagenome/lib/python3.9/_compression.py”, line 103, in read
data = self._decompressor.decompress(rawblock, size)
OSError: Invalid data stream
1.python version : 3.7.3.final.0
2.virtual packages : __linux=3.10.0=0
__glibc=2.17=0
__unix=0=0
__archspec=1=x86_64
3.MetaPhlAn version 4.0.4 (17 Jan 2023)
4.PhyloPhlAn version 3.0.67 (24 August 2022)
I have the same issue. it looks like some wrong with the file ‘mpa_vOct22_CHOCOPhlAnSGB_202212.pkl’.
When I read this file according to the source code, I get an error. “OSError: Invalid data stream”. And the last version ‘mpa_vJan21_CHOCOPhlAnSGB_202103.pkl’ is good.
So you can try previous version database for test.
Hi @Richard_Liu@licui
Thanks for reporting this problem. I apologise for the inconvenience. Indeed, we have just detected that the pkl file uploaded to our servers had been corrupted. I just re-uploaded all the data and retested the databases again and all seem to be working. Please, try to re-download the database again and let me know if you encounter any other problem.
Was struggling with this OSError: Invalid data stream error all day yesterday…
Just started a fresh conda environment (conda create -n metaphlan4), and installed metaphlan (conda install -c bioconda metaphlan). Tried to install the database, got this:
Traceback (most recent call last):
File “/usr/local/bin/miniconda3/envs/metaphlan4/bin/metaphlan”, line 10, in
sys.exit(main())
File “/usr/local/bin/miniconda3/envs/metaphlan4/lib/python3.6/site-packages/metaphlan/metaphlan.py”, line 1187, in main
pars[‘index’] = check_and_install_database(pars[‘index’], pars[‘bowtie2db’], pars[‘bowtie2_build’], pars[‘nproc’], pars[‘force_download’])
File “/usr/local/bin/miniconda3/envs/metaphlan4/lib/python3.6/site-packages/metaphlan/metaphlan.py”, line 589, in check_and_install_database
index = resolve_latest_database(bowtie2_db, ls_f[‘mpa_latest’], force_redownload_latest)
UnboundLocalError: local variable ‘ls_f’ referenced before assignment
Same error occurs if I use metaphlan --install or run a full metaphlan analysis command line.
Should it be trying to download something from Dropbox?
Hi @MikeC
It seems your system is trying to install an old version of metaphlan (probably version 3) you can check the version by running $ metaphlan --version
Still something wrong. Cleaned out prior database files at /usr/local/bin/miniconda3/envs/metap/lib/python3.7/site-packages/metaphlan/metaphlan_databases, and re-ran install:
Warning: Unable to extract /usr/local/bin/miniconda3/envs/metap/lib/python3.7/site-packages/metaphlan/metaphlan_databases/mpa_vOct22_CHOCOPhlAnSGB_202212_bt2.tar.
File “/usr/local/bin/miniconda3/envs/metap/lib/python3.7/site-packages/metaphlan/metaphlan_databases/mpa_vOct22_CHOCOPhlAnSGB_202212.md5” not found!
File “/usr/local/bin/miniconda3/envs/metap/lib/python3.7/site-packages/metaphlan/metaphlan_databases/mpa_vOct22_CHOCOPhlAnSGB_202212.tar” not found!
MD5 checksums not found, something went wrong!
Not a disk space issue, I have terabytes of free space. The (corrupted) databases I downloaded yesterday using the same command worked fine.
Stand by – I might have a disk space issue after all… Installation was mistakenly pointing to a 1 Tb SSD drive rather than my much larger data partition…
Deleted everything and started over, from the installation of miniconda.
I did have to include the conda-forge channel to get MetaPhlAn version 4.0.5 (23 Feb 2023).
Installed the databases without issue.
Ran a test analysis on a metagenome, and metaphlan completed without errors.
With the new database, I found a bug in the Humann3 config.py . You set metaphlan_v4_db_version=“vJan21” , but latest database names “vOct”. And error likes
config.metaphlan_v3_db_version+" or “+metaphlan_v4_db_version+” . Please update your version of MetaPhlAn to at least v3.0."
NameError: name ‘metaphlan_v4_db_version’ is not defined
I guess this might be a bug and for your reference
Hi @licui
The current version of Humann 3.6 is still not compatible with the latest version of the metaphlan database (vOct22). My suggestion will be to keep them separately in two different environments, one keeping humann 3.6 with vJan21 for humann profiling and a separate one with metaphlan with vOct22 if you are interested on having the tax profiles with the latest database
hello, i am running humann3 but failed many times. i download metaphlan database(vJan21) use wget commond in the metaphlan installation dir, couse you said that Humann 3.6 is still not compatible with the latest version of the metaphlan database (vOct22). but when i try human3 again, it still auto download the database and error again. belows is my database content,
compared with database download by metaphlan --install --bowtie2db command. so can you tell me how to resolve this problem? or i want konw how to use metaphlan --install --bowtie2db to donwnload the old version database. thank you very much！