# Meet "OSError: Invalid data stream" when using panphlan\_profiling.py

**URL:** <https://forum.biobakery.org/t/meet-oserror-invalid-data-stream-when-using-panphlan-profiling-py/1277>\
**Category:** PanPhlAn\
**Created:** [November 14, 2020, 7:24pm UTC](https://forum.biobakery.org/t/meet-oserror-invalid-data-stream-when-using-panphlan-profiling-py/1277 "2020-11-14T19:24:26Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![wangzhenyu](https://avatars.discourse-cdn.com/v4/letter/w/49beb7/32.png) [@wangzhenyu](https://forum.biobakery.org/u/wangzhenyu)\
**Post date:** [November 14, 2020, 7:24pm UTC](https://forum.biobakery.org/t/meet-oserror-invalid-data-stream-when-using-panphlan-profiling-py/1277/1 "2020-11-14T19:24:26Z")

</div>

I am new to panphlan3 and i meet error when i used panphlan\_profiling.py.  
The code i perform is as following:  
(panphlan) wangzhenyu@wangzhenyudeMacBook-Pro panphlan % panphlan\_profiling.py   
-i ./map\_results/   
-p ./Bifidobacterium\_pseudocatenulatum/Bifidobacterium\_pseudocatenulatum\_pangenome.tsv   
–o\_matrix ./profiling\_results/matrix.csv

STEP 1. Processing genes informations from pangenome file…  
Number of reference genomes: 18  
Average number of gene-families per genome: 1761  
Total number of pangenome gene-families 4550

STEP 2. Create coverage matrix  
Traceback (most recent call last):  
File “/Users/wangzhenyu/opt/miniconda3/envs/panphlan/bin/panphlan\_profiling.py”, line 932, in   
main()  
File “/Users/wangzhenyu/opt/miniconda3/envs/panphlan/bin/panphlan\_profiling.py”, line 864, in main  
dna\_samples\_covs = read\_map\_results(args.i\_dna, args.verbose)  
File “/Users/wangzhenyu/opt/miniconda3/envs/panphlan/bin/panphlan\_profiling.py”, line 299, in read\_map\_results  
dna\_samples\_covs[dna\_sample\_id] = read\_gene\_cov\_file(os.path.join(i\_dna, dna\_covs\_file))  
File “/Users/wangzhenyu/opt/miniconda3/envs/panphlan/bin/panphlan\_profiling.py”, line 285, in read\_gene\_cov\_file  
for line in f:  
File “/Users/wangzhenyu/opt/miniconda3/envs/panphlan/lib/python3.7/bz2.py”, line 215, in readline  
return self.\_buffer.readline(size)  
File “/Users/wangzhenyu/opt/miniconda3/envs/panphlan/lib/python3.7/\_compression.py”, line 68, in readinto  
data = self.read(len(byte\_view))  
File “/Users/wangzhenyu/opt/miniconda3/envs/panphlan/lib/python3.7/\_compression.py”, line 103, in read  
data = self.\_decompressor.decompress(rawblock, size)  
OSError: Invalid data stream

Do anyone also meet this problem and know how to solve it?  
Thanks for viewing and answering.

---

<div class="post-metadata">

**Author:** ![PabloGallardoM](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/pablogallardom/32/1471_2.png) [@PabloGallardoM](https://forum.biobakery.org/u/PabloGallardoM)\
**Post date:** [October 21, 2022, 11:26am UTC](https://forum.biobakery.org/t/meet-oserror-invalid-data-stream-when-using-panphlan-profiling-py/1277/2 "2022-10-21T11:26:25Z")

</div>

did you already solve this? It happens to me also.

---

<div class="post-metadata">

**Author:** ![leonard.dubois](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/leonard.dubois/32/750_2.png) [@leonard.dubois](https://forum.biobakery.org/u/leonard.dubois)\
**Post date:** [October 21, 2022, 1:11pm UTC](https://forum.biobakery.org/t/meet-oserror-invalid-data-stream-when-using-panphlan-profiling-py/1277/3 "2022-10-21T13:11:58Z")

</div>

Hi,

what’s the content of your `map_results/` folder ? It should be only compressed output files of `panphlan_mapping.py`. The profiling, if provided with a folder path in the `--input` parameter, will consider all the files in it. It might be that one of them is not a mapping output and mess up with the reading process that expect compression.

To avoid that, you can provide a text file with the list of path to the mapping results in the panphlan\_profiling.py --input parameter. I usually go for that setup to avoid issue.

Hope this can help

---

<div class="post-metadata">

**Author:** ![PabloGallardoM](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/pablogallardom/32/1471_2.png) [@PabloGallardoM](https://forum.biobakery.org/u/PabloGallardoM)\
**Post date:** [October 23, 2022, 9:28am UTC](https://forum.biobakery.org/t/meet-oserror-invalid-data-stream-when-using-panphlan-profiling-py/1277/4 "2022-10-23T09:28:28Z")

</div>

Hi, thanks for your reply, the output is only tsv.bz2 files, but still having this issue.

STEP 2. Create coverage matrix  
[I] Reading mapping result file: Sample2.Ecoli.tsv.bz2  
[I] Reading mapping result file: .DS\_Store  
Traceback (most recent call last):  
File “/Users/arrow/miniconda3/bin/panphlan\_profiling.py”, line 763, in   
main()  
File “/Users/arrow/miniconda3/bin/panphlan\_profiling.py”, line 709, in main  
dna\_samples\_covs = read\_map\_results(args.i\_dna, args.verbose)  
File “/Users/arrow/miniconda3/bin/panphlan\_profiling.py”, line 286, in read\_map\_results  
dna\_samples\_covs[dna\_sample\_id] = read\_gene\_cov\_file(os.path.join(i\_dna, dna\_covs\_file))  
File “/Users/arrow/miniconda3/bin/panphlan\_profiling.py”, line 272, in read\_gene\_cov\_file  
for line in f:  
File “/Users/arrow/miniconda3/lib/python3.9/bz2.py”, line 208, in readline  
return self.\_buffer.readline(size)  
File “/Users/arrow/miniconda3/lib/python3.9/\_compression.py”, line 68, in readinto  
data = self.read(len(byte\_view))  
File “/Users/arrow/miniconda3/lib/python3.9/\_compression.py”, line 103, in read  
data = self.\_decompressor.decompress(rawblock, size)  
OSError: Invalid data stream

I will try your idea.  
thank you again. 🙂

Pablo