# PhyloPhlAn: using a manually downloaded database

**URL:** <https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979>\
**Category:** PhyloPhlAn\
**Created:** [April 21, 2021, 5:12pm UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979 "2021-04-21T17:12:10Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![confectioners](https://avatars.discourse-cdn.com/v4/letter/c/45deac/32.png) [@confectioners](https://forum.biobakery.org/u/confectioners)\
**Post date:** [April 21, 2021, 5:12pm UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/1 "2021-04-21T17:12:10Z")

</div>

Dear PhyloPhlAn team and community,

Having run this command in PhyloPhlAn (version 3.0.60 (27 November 2020), installed through conda):  
(myenv) usr@srvr:~$ phylophlan -i /home/usr/data/input/11assemblies -d /home/usr/phylophlan\_databases phylophlan\_databases --diversity medium -f supermatrix\_nt.cfg --nproc 8

…resulted in the following:  
Traceback (most recent call last):  
File “/home/usr/miniconda3/envs/myenv/bin/phylophlan”, line 10, in   
sys.exit(phylophlan\_main())  
File “/home/usr/miniconda3/envs/myenv/lib/python3.7/site-packages/phylophlan/phylophlan.py”, line 3227, in phylophlan\_main  
verbose=args.verbose)  
File “/home/usr/miniconda3/envs/myenv/lib/python3.7/site-packages/phylophlan/phylophlan.py”, line 818, in init\_database  
for f in glob.iglob(os.path.join(folder, ‘\*’))  
File “/home/usr/miniconda3/envs/myenv/lib/python3.7/site-packages/phylophlan/phylophlan.py”, line 819, in   
for \_, seq in SimpleFastaParser(bz2.open(f, ‘rt’) if f.endswith(’.bz2’) else open(f))])  
File “/home/usr/miniconda3/envs/myenv/lib/python3.7/site-packages/Bio/SeqIO/FastaIO.py”, line 47, in SimpleFastaParser  
for line in handle:  
File “/home/usr/miniconda3/envs/myenv/lib/python3.7/codecs.py”, line 322, in decode  
(result, consumed) = self.\_buffer\_decode(data, self.errors, final)  
UnicodeDecodeError: ‘utf-8’ codec can’t decode byte 0xba in position 1035: invalid start byte  
(myenv) usr@srvr:~$

I would greatly appreciate any feedback on how to fix this issue, so I could run PhyloPhlAn to construct a phylogeny of 11 whole-genome assemblies from pure cultured strains (no metagenomic data).

These assemblies are in this folder:  
/home/usr/data/input/11assemblies

The database is in this folder:  
/home/usr/phylophlan\_databases  
containing the following 2 files (i) and (ii), both manually downloaded via [http://cmprod1.cibio.unitn.it/databases/PhyloPhlAn/phylophlan\_databases.txt:](http://cmprod1.cibio.unitn.it/databases/PhyloPhlAn/phylophlan_databases.txt:)  
(i) phylophlan.tar  
(downloaded from [https://zenodo.org/record/4005620/files/phylophlan.tar?download=1](https://zenodo.org/record/4005620/files/phylophlan.tar?download=1))  
(ii) phylophlan.md5  
(downloaded from [https://zenodo.org/record/4005620/files/phylophlan.md5?download=1](https://zenodo.org/record/4005620/files/phylophlan.md5?download=1))

Please note that I had to download the database manually (following suggestions found here: [Using manually downloaded database · Issue #18 · biobakery/phylophlan · GitHub](https://github.com/biobakery/phylophlan/issues/18#issuecomment-635334776), “Using manually downloaded database #18”) because of limitations with the internet connection/firewall on my system.

Also, I have already come across this conversation: [local variable 'input\_faa\_clean' referenced before assignment · Issue #9 · biobakery/phylophlan · GitHub](https://github.com/biobakery/phylophlan/issues/9#issuecomment-621043051) (“local variable ‘input\_faa\_clean’ referenced before assignment #9”), suggesting that getting PhyloPhlAn directly from the repository would fix an issue which I guess is similar to mine (if not identical). Unfortunately, I can’t install PhyloPhlAn directly from the repository because of limitations with the internet connection/firewall on my system.

I would be very happy for any suggestions what I could do in order to get PhyloPhlAn running?

Thanks already in advance,  
Michael

---

<div class="post-metadata">

**Author:** ![f.asnicar](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/f.asnicar/32/196_2.png) [@f.asnicar](https://forum.biobakery.org/u/f.asnicar)\
**Post date:** [April 26, 2021, 8:11am UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/2 "2021-04-26T08:11:49Z")

</div>

Hello Michael, sorry I didn’t receive the notification of your post.

I think the problem is that you’re specifying the folder containing the database(s) with the param you should specify which database to use from that folder.

The command:  
`phylophlan -i /home/usr/data/input/11assemblies -d /home/usr/phylophlan_databases phylophlan_databases --diversity medium -f supermatrix_nt.cfg --nproc 8`

should be:  
`phylophlan -i /home/usr/data/input/11assemblies -d phylophlan --databases_folder /home/usr/phylophlan_databases phylophlan_databases --diversity medium -f supermatrix_nt.cfg --nproc 8`

Please let me know if something is not clear.

Many thanks,  
Francesco

---

<div class="post-metadata">

**Author:** ![confectioners](https://avatars.discourse-cdn.com/v4/letter/c/45deac/32.png) [@confectioners](https://forum.biobakery.org/u/confectioners)\
**Post date:** [April 27, 2021, 4:52pm UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/3 "2021-04-27T16:52:16Z")

</div>

Hello Francesco,

Thank you very much for your reply and analysis of the problem.

Having used your command and, based on it, slightly modified commands, I have unfortunately not yet succeeded in getting PhyloPhlAn running - but I hope to have gotten closer.

Below, I have included details about these commands, hoping that this may help in narrowing down the problem. (I have decided to split this information into separate posts so it is easier to grasp.)

Any feedback on this would be greatly appreciated.

Many thanks,  
Michael

---

<div class="post-metadata">

**Author:** ![confectioners](https://avatars.discourse-cdn.com/v4/letter/c/45deac/32.png) [@confectioners](https://forum.biobakery.org/u/confectioners)\
**Post date:** [April 27, 2021, 4:53pm UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/4 "2021-04-27T16:53:25Z")

</div>

**(1)** With your command (as suggested under ‘should be:’ in your post)…  
phylophlan  
-i /home/usr/data/input/11assemblies  
-d phylophlan  
–databases\_folder /home/usr/phylophlan\_databases phylophlan\_databases  
–diversity medium  
-f supermatrix\_nt.cfg  
–nproc 8

…I got the following:  
usage: phylophlan [-h] [-i INPUT | -c CLEAN] [-o OUTPUT] [-d DATABASE]  
[-t {n,a}] [-f CONFIG\_FILE] --diversity {low,medium,high}  
[–accurate | --fast] [–clean\_all] [–database\_list]  
[-s SUBMAT] [–submat\_list] [–submod\_list] [–nproc NPROC]  
[–min\_num\_proteins MIN\_NUM\_PROTEINS]  
[–min\_len\_protein MIN\_LEN\_PROTEIN]  
[–min\_num\_markers MIN\_NUM\_MARKERS]  
[–trim {gap\_trim,gap\_perc,not\_variant,greedy}]  
[–gap\_perc\_threshold GAP\_PERC\_THRESHOLD]  
[–not\_variant\_threshold NOT\_VARIANT\_THRESHOLD]  
[–subsample {phylophlan,onethousand,sevenhundred,fivehundred,threehundred,onehundred,fifty,twentyfive,tenpercent,twentyfivepercent,fiftypercent,full}]  
[–unknown\_fraction UNKNOWN\_FRACTION]  
[–scoring\_function {trident,muscle,random}] [–sort]  
[–remove\_fragmentary\_entries]  
[–fragmentary\_threshold FRAGMENTARY\_THRESHOLD]  
[–min\_num\_entries MIN\_NUM\_ENTRIES] [–maas MAAS]  
[–remove\_only\_gaps\_entries] [–mutation\_rates]  
[–force\_nucleotides] [–input\_folder INPUT\_FOLDER]  
[–data\_folder DATA\_FOLDER]  
[–databases\_folder DATABASES\_FOLDER]  
[–submat\_folder SUBMAT\_FOLDER]  
[–submod\_folder SUBMOD\_FOLDER]  
[–configs\_folder CONFIGS\_FOLDER]  
[–output\_folder OUTPUT\_FOLDER]  
[–genome\_extension GENOME\_EXTENSION]  
[–proteome\_extension PROTEOME\_EXTENSION] [–update]  
[–citation] [–verbose] [-v]  
phylophlan: error: unrecognized arguments: phylophlan\_databases

---

<div class="post-metadata">

**Author:** ![confectioners](https://avatars.discourse-cdn.com/v4/letter/c/45deac/32.png) [@confectioners](https://forum.biobakery.org/u/confectioners)\
**Post date:** [April 27, 2021, 4:54pm UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/5 "2021-04-27T16:54:00Z")

</div>

**(2)** Thus, I removed “phylophlan\_databases” from this command, resulting in the following command…  
phylophlan  
-i /home/usr/data/input/11assemblies  
-d phylophlan  
–databases\_folder /home/usr/phylophlan\_databases  
–diversity medium  
-f supermatrix\_nt.cfg  
–nproc 8

…for which I got the following:  
[e] -t/–db\_type not specified and could not automatically detect the input database file(s)

---

<div class="post-metadata">

**Author:** ![confectioners](https://avatars.discourse-cdn.com/v4/letter/c/45deac/32.png) [@confectioners](https://forum.biobakery.org/u/confectioners)\
**Post date:** [April 27, 2021, 4:56pm UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/6 "2021-04-27T16:56:55Z")

</div>

**(3)** Both “-t” and “–db\_type” are listed as optional arguments in the PhyloPhlAn user manual (version Jun 3, 2020, section `phylophlan.py`), so I added them to the previous command, resulting in the following command…  
phylophlan  
-i /home/usr/data/input/11assemblies  
-d phylophlan  
-t n  
–db\_type n  
–databases\_folder /home/usr/phylophlan\_databases  
–diversity medium  
-f supermatrix\_nt.cfg  
–nproc 8

…for which I got the following:  
[e] database format ("/home/usr/phylophlan\_databases/phylophlan/phylophlan.fna", “/home/usr/phylophlan\_databases/phylophlan/phylophlan.fna.bz2”, or “/home/usr/phylophlan\_databases/phylophlan”) not recognize

Please note that the folder  
/home/usr/phylophlan\_databases  
contains the following 2 files:  
phylophlan.tar  
and  
phylophlan.md5

To be more precise, these file paths are:  
/home/usr/phylophlan\_databases/phylophlan.tar  
and  
/home/usr/phylophlan\_databases/phylophlan.md5

---

<div class="post-metadata">

**Author:** ![confectioners](https://avatars.discourse-cdn.com/v4/letter/c/45deac/32.png) [@confectioners](https://forum.biobakery.org/u/confectioners)\
**Post date:** [April 27, 2021, 5:00pm UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/7 "2021-04-27T17:00:54Z")

</div>

P.S.: Please let me know if any further information would be required to help solve this problem and get PhyloPhlAn running in my conda environment.

Thank you very much again,  
Michael

---

<div class="post-metadata">

**Author:** ![f.asnicar](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/f.asnicar/32/196_2.png) [@f.asnicar](https://forum.biobakery.org/u/f.asnicar)\
**Post date:** [April 28, 2021, 11:45am UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/8 "2021-04-28T11:45:55Z")

</div>

Hello Michael,

Thanks for your messages. Sorry for the wrong command, definitely was a copy-paste mistake. The command you corrected:

> [@confectioners](#):
>
> ```auto
> phylophlan -i /home/usr/data/input/11assemblies -d phylophlan --databases_folder /home/usr/phylophlan_databases --diversity medium -f supermatrix_nt.cfg --nproc 8
> 
> ```

looks good.

The error:

> [@confectioners](#):
>
> [e] -t/–db\_type not specified and could not automatically detect the input database file(s)

Is a bit strange as PhyloPhlAn should be able to automatically infer it from the database file(s).  
Your fix `-t n` (no need for `--db_type` and `-t` is its short version) doesn’t work because the `phylophlan` database is a set of proteins, so you should specify `-t a`.

Will you be able to run this command (which should be correct now):

```auto
phylophlan -i /home/usr/data/input/11assemblies -d phylophlan -t a --databases_folder /home/usr/phylophlan_databases --diversity medium -f supermatrix_aa.cfg --nproc 8 --verbose 2>&1 | tee phylophlan_11assemblies.log

```

That should generate a log file `phylophlan_11assemblies.log` which would be great if you can share it with me if you still run into errors.

Many thanks,  
Francesco

---

<div class="post-metadata">

**Author:** ![confectioners](https://avatars.discourse-cdn.com/v4/letter/c/45deac/32.png) [@confectioners](https://forum.biobakery.org/u/confectioners)\
**Post date:** [April 28, 2021, 4:04pm UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/9 "2021-04-28T16:04:43Z")

</div>

Hello Francesco,

Thanks a lot for your reply and efforts in further analyzing this problem.

Happy to hear that my command (quoted by you) looks good.

As you noted, I was also astonished that PhyloPhlAn should be able to automatically detect the database file(s), assuming that it would be sufficient to download the database files (phylophlan.tar and phylophlan.md5), put them in a folder and indicate the location of this folder using the “–databases\_folder” command.

Also thanks for having explained and corrected my rookie mistake with the selection of “-t n” (wrong) instead of “-t a”.

Having run the command which you have kindly included in your reply, I still get an error:  
[e] database format ("/home/usr/phylophlan\_databases/phylophlan/phylophlan.faa", “/home/usr/phylophlan\_databases/phylophlan/phylophlan.faa.bz2”, or “/home/usr/phylophlan\_databases/phylophlan”) not recognize

Also, I have attached the phylophlan\_11assemblies.log, and I am sorry that I have to come back to your kind offer sharing it with you (Biobakery wouldn’t let me upload a \*.log file (“Sorry, the file you are trying to upload is not authorized (authorized extensions: jpg, jpeg, png, gif, txt, rtf, csv, tsv, biom).”), so I have changed the file extension to \*.txt).

Many thanks again,  
Michael[phylophlan\_11assemblies.txt](https://forum.biobakery.org/uploads/short-url/kqNJkUCdOgXn3OvSEkTvFPKMM0c.txt) (2.5 KB)

---

<div class="post-metadata">

**Author:** ![confectioners](https://avatars.discourse-cdn.com/v4/letter/c/45deac/32.png) [@confectioners](https://forum.biobakery.org/u/confectioners)\
**Post date:** [April 28, 2021, 4:07pm UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/10 "2021-04-28T16:07:23Z")

</div>

P.S.:  
This is the command and what I got from the Terminal:

(base) usr@srvr:~$ conda activate myenv  
(myenv) usr@srvr:~$ phylophlan -i /home/usr/data/input/11assemblies -d phylophlan -t a --databases\_folder /home/usr/phylophlan\_databases --diversity medium -f supermatrix\_aa.cfg --nproc 8 --verbose 2\>&1 | tee phylophlan\_11assemblies.log  
PhyloPhlAn version 3.0.60 (27 November 2020)

Command line: /home/usr/miniconda3/envs/myenv/bin/phylophlan -i /home/usr/data/input/11assemblies -d phylophlan -t a --databases\_folder /home/usr/phylophlan\_databases --diversity medium -f supermatrix\_aa.cfg --nproc 8 --verbose

Automatically setting “input=11assemblies” and “input\_folder=/home/usr/data/input”  
“medium-accurate” preset  
Setting “sort=True” because “database=phylophlan”  
Setting “min\_num\_markers=100” since no value has been specified and the “database=phylophlan”  
Arguments: {‘input’: ‘11assemblies’, ‘clean’: None, ‘output’: ‘11assemblies\_phylophlan’, ‘database’: ‘phylophlan’, ‘db\_type’: ‘a’, ‘config\_file’: ‘supermatrix\_aa.cfg’, ‘diversity’: ‘medium’, ‘accurate’: True, ‘fast’: False, ‘clean\_all’: False, ‘database\_list’: False, ‘submat’: ‘pfasum60’, ‘submat\_list’: False, ‘submod\_list’: False, ‘nproc’: 8, ‘min\_num\_proteins’: 1, ‘min\_len\_protein’: 50, ‘min\_num\_markers’: 100, ‘trim’: ‘gap\_trim’, ‘gap\_perc\_threshold’: 0.67, ‘not\_variant\_threshold’: 0.99, ‘subsample’: \<function onehundred at 0x7f2f8cd8a3b0\>, ‘unknown\_fraction’: 0.3, ‘scoring\_function’: \<function trident at 0x7f2f8cd8a710\>, ‘sort’: True, ‘remove\_fragmentary\_entries’: False, ‘fragmentary\_threshold’: 0.85, ‘min\_num\_entries’: 4, ‘maas’: None, ‘remove\_only\_gaps\_entries’: False, ‘mutation\_rates’: False, ‘force\_nucleotides’: False, ‘input\_folder’: ‘/home/usr/data/input/11assemblies’, ‘data\_folder’: ‘11assemblies\_phylophlan/tmp’, ‘databases\_folder’: ‘/home/usr/phylophlan\_databases’, ‘submat\_folder’: ‘/home/usr/miniconda3/envs/myenv/lib/python3.7/site-packages/phylophlan/phylophlan\_substitution\_matrices/’, ‘submod\_folder’: ‘/home/usr/miniconda3/envs/myenv/lib/python3.7/site-packages/phylophlan/phylophlan\_substitution\_models/’, ‘configs\_folder’: ‘/home/usr/miniconda3/envs/myenv/lib/python3.7/site-packages/phylophlan/phylophlan\_configs/’, ‘output\_folder’: ‘’, ‘genome\_extension’: ‘.fna’, ‘proteome\_extension’: ‘.faa’, ‘update’: False, ‘verbose’: True}  
Loading configuration file “supermatrix\_aa.cfg”  
Checking configuration file  
Checking “/home/usr/miniconda3/envs/myenv/bin/diamond”  
Checking “/home/usr/miniconda3/envs/myenv/bin/mafft”  
Checking “/home/usr/miniconda3/envs/myenv/bin/trimal”  
Checking “/home/usr/miniconda3/envs/myenv/bin/FastTreeMP”  
Checking “/home/usr/miniconda3/envs/myenv/bin/raxmlHPC-PTHREADS-SSE3”  
[e] database format ("/home/usr/phylophlan\_databases/phylophlan/phylophlan.faa", “/home/usr/phylophlan\_databases/phylophlan/phylophlan.faa.bz2”, or “/home/usr/phylophlan\_databases/phylophlan”) not recognize  
(myenv) usr@srvr:~$

---

<div class="post-metadata">

**Author:** ![confectioners](https://avatars.discourse-cdn.com/v4/letter/c/45deac/32.png) [@confectioners](https://forum.biobakery.org/u/confectioners)\
**Post date:** [April 28, 2021, 4:10pm UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/11 "2021-04-28T16:10:55Z")

</div>

P.P.S.:  
Just for your information: I have replaced the real names of the server, the user and the input folder with “srvr”, “usr” and “11assemblies”, respectively. However, all of the real names only consist of letters (small and big), numbers and underscores, no whitespace or points, and they are not overly long or complicated either.

---

<div class="post-metadata">

**Author:** ![confectioners](https://avatars.discourse-cdn.com/v4/letter/c/45deac/32.png) [@confectioners](https://forum.biobakery.org/u/confectioners)\
**Post date:** [April 28, 2021, 4:30pm UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/12 "2021-04-28T16:30:12Z")

</div>

P.P.P.S.:  
(a)  
The **configuration files** were created by running the phylophlan\_write\_default\_configs.sh and are located in /home/usr/  
e.g.  
/home/usr/supermatrix\_nt.cfg

For my phylogeny, I have chosen the Supermatrix pipeline rather than the Supertree pipeline (as recommended in the PhyloPhlAn user manual (version Jun 3, 2020, `supermatrix-or-supertree-approach`).  
Also, I have chosen ‘supermatrix\_nt.cfg’ rather than ‘supermatrix\_aa.cfg’ because my input consists of whole-genome assemblies (nucleotide sequences) rather than translated protein sequences.

(b)  
The **assemblies** in the /home/usr/data/input/11assemblies folder are mostly publicly available whole-genome assemblies downloaded from GenBank, such as e.g.  
A\_prevotii\_GCF\_000024105.1\_ASM2410v1\_genomic.fna

(downloaded from  
NCBI Assembly  
as  
GCF\_000024105.1\_ASM2410v1\_genomic.fna  
and renamed to  
A\_prevotii\_GCF\_000024105.1\_ASM2410v1\_genomic.fna)

---

<div class="post-metadata">

**Author:** ![f.asnicar](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/f.asnicar/32/196_2.png) [@f.asnicar](https://forum.biobakery.org/u/f.asnicar)\
**Post date:** [April 29, 2021, 8:35am UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/13 "2021-04-29T08:35:11Z")

</div>

Hello Michael!

I think I found the problem, thank you for your patience and for reporting this.

With the latest commit in the GitHub repository, this should be now fixed: [GitHub - biobakery/phylophlan: Precise phylogenetic analysis of microbial isolates and genomes from metagenomes](https://github.com/biobakery/phylophlan)  
From the repo, you can download the [`phylophlan.py`](https://github.com/biobakery/phylophlan/blob/7bbe8a7d31800a93f4d53982d6a47a3c03ea47c5/phylophlan/phylophlan.py) Python script and you can run it (with your phylophlan conda env activated) with `./phylophlan.py` and use the same params as above.

Please, let me know if this fix is working for you.

I just wanted to add a small note about the config file:

> [@confectioners](#):
>
> Also, I have chosen ‘supermatrix\_nt.cfg’ rather than ‘supermatrix\_aa.cfg’ because my input consists of whole-genome assemblies (nucleotide sequences) rather than translated protein sequences.

The `nt` or `aa` in the config filename actually refers to the database, so, if you’re gonna use the `phylophlan` database you’ll need the `supermatrix_aa.cfg` config file which contains the instructions for indexing an amino acids database and also the commands for the translated search, since your inputs are genomes. By default this config will then build a phylogeny based on the AAs MSA, instead, if you want the phylogeny to be built on nucleotides, you would specify the `--force_nucleotides` param when both creating the config file and when running PhyloPhlAn. Note that this will only work if your inputs are all genomes and it won’t work if your inputs are a mix of genomes and proteomes.

Many thanks,  
Francesco

---

<div class="post-metadata">

**Author:** ![confectioners](https://avatars.discourse-cdn.com/v4/letter/c/45deac/32.png) [@confectioners](https://forum.biobakery.org/u/confectioners)\
**Post date:** [April 29, 2021, 2:37pm UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/14 "2021-04-29T14:37:21Z")

</div>

Hello Francesco,

Thank you very much again for your quick reply and efforts!

I’ll get back to you as soon as I’ve checked if the fix was working.

Many thanks,  
Michael

---

<div class="post-metadata">

**Author:** ![confectioners](https://avatars.discourse-cdn.com/v4/letter/c/45deac/32.png) [@confectioners](https://forum.biobakery.org/u/confectioners)\
**Post date:** [April 30, 2021, 11:18am UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/15 "2021-04-30T11:18:26Z")

</div>

Hello Francesco,

Having run the phylophlan.py script, there is still a (little) issue.

Running this command…

> [@confectioners](#):
>
> `phylophlan -i /home/usr/data/input/11assemblies -d phylophlan --databases_folder /home/usr/phylophlan_databases --diversity medium -f supermatrix_nt.cfg --nproc 8`

…I got the following:  
(base) usr@ srvr:~$ conda activate myenv  
(myenv) usr@ srvr:~$ phylophlan -i /home/usr/data/input/11assemblies -d phylophlan --databases\_folder /home/usr/phylophlan\_databases --diversity medium -f supermatrix\_nt.cfg --nproc 8  
[e] both db\_dna and db\_aa are None!  
(myenv) usr@srvr:~$

Also, I have noticed that the folder in which I have put the manually downloaded database including the checksum file  
/home/usr/phylophlan\_databases/phylophlan.tar  
/home/usr/phylophlan\_databases/phylophlan.md5

now contains another folder  
/home/usr/phylophlan\_databases/phylophlan

with another zipped file  
/home/usr/phylophlan\_databases/phylophlan/phylophlan.faa.bz2

Would be great if you could let me know if you see where the problem is.

Many thanks,  
Michael

---

<div class="post-metadata">

**Author:** ![f.asnicar](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/f.asnicar/32/196_2.png) [@f.asnicar](https://forum.biobakery.org/u/f.asnicar)\
**Post date:** [April 30, 2021, 12:22pm UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/16 "2021-04-30T12:22:11Z")

</div>

Hi Michael,

Thanks, that’s great. The folder is the actual database folder, and it should contain the zip file, the same file uncompressed and it indexed version with the mapping software specified in the configuration file.

The error you got now:

> [@confectioners](#):
>
> [e] both db\_dna and db\_aa are None!

is likely because the config file does not contain the proper sections for generating the indexed version of the database. From the filename, it seems you’re using the default config file for a nucleotide/genes database (`supermatrix_nt.cfg`), so that might explain the error. Since the `phylophlan` database is a set of 400 universally conserved proteins, you should use the default config file generated for a protein database: `supermatrix_aa.cfg`.  
If you don’t have such a config file it should be pretty easy to generate one with the `phylophlan_write_config_file` utility.

Please let me know if something Is not clear.

Many thanks,  
Francesco

---

<div class="post-metadata">

**Author:** ![confectioners](https://avatars.discourse-cdn.com/v4/letter/c/45deac/32.png) [@confectioners](https://forum.biobakery.org/u/confectioners)\
**Post date:** [April 30, 2021, 6:09pm UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/17 "2021-04-30T18:09:19Z")

</div>

Hi Francesco,

Thank you very much again for your swift and patient reply.

Sorry for having overread the explanations about the nt vs. aa databases you had included already in your previous answer, and it makes totally sense to use the protein database as I want PhyloPhlAn to calculate a protein (not nucleotide) based phylogenomic tree.

Most importantly, with this little correction the command was working and PhyloPhlAn is running. 🙂

Grazie mille and many thanks again,  
Michael

---

<div class="post-metadata">

**Author:** ![Keaton\_Stagaman1](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/keaton_stagaman1/32/2675_2.png) [@Keaton\_Stagaman1](https://forum.biobakery.org/u/Keaton_Stagaman1)\
**Post date:** [January 24, 2024, 1:59am UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/18 "2024-01-24T01:59:23Z")

</div>

I seem to be having a similar problem, but the solutions in the thread haven’t worked for me. I’m working on a cluster with no internet access, so I downloaded the phylophlan database manually and copied them into /home/phylophlan. I generated the config file as such:

```auto
phylophlan_write_config_file \
    -o custom_config_aa.cfg \
    -d a \
    --db_aa diamond \
    --map_aa diamond \
    --map_dna diamond \
    --msa muscle \
    --trim trimal \
    --tree1 fasttree \
    --tree2 raxml \
    --force_nucleotides

```

and I then executed the following command:

```auto
phylophlan \
    -i /PATH/TO/MAGS/ \
    -o /PATH/TO/MAGS/phylophlan_out \
    --genome_extension ".fasta" \
    -d phylophlan \
    --databases_folder /home/phylophlan \
    -t a \
    --diversity medium \
    -f custom_config_aa.cfg \
    --nproc 20 \
    --force_nucleotides \
    --verbose

```

However, I get the following output and error:

```auto
PhyloPhlAn version 3.0.67 (24 August 2022)

Command line: /home/bin/phylophlan -i /PATH/TO/MAGS/ -o /PATH/TO/MAGS/phylophlan_out --genome_extension .fasta -d phylophlan --databases_folder /home/phylophlan -t a --diversity medium -f custom_config_aa.cfg --nproc 20 --force_nucleotides --verbose

Automatically setting "input=refined" and "input_folder=/PATH/TO/MAGS/"
"medium-accurate" preset
Setting "sort=True" because "database=phylophlan"
Setting "min_num_markers=100" since no value has been specified and the "database=phylophlan"

Arguments: {'input': 'MAGS', 'clean': None, 'output': '/PATH/TO/MAGS/phylophlan_out', 'database': 'phylophlan', 'db_type': 'a', 'config_file': 'phylophlan_configs/custom_config_aa.cfg', 'diversity': 'medium', 'accurate': True, 'fast': False, 'clean_all': False, 'database_list': False, 'submat': 'pfasum60', 'submat_list': False, 'submod_list': False, 'nproc': 20, 'min_num_proteins': 1, 'min_len_protein': 50, 'min_num_markers': 100, 'trim': 'gap_trim', 'gap_perc_threshold': 0.67, 'not_variant_threshold': 0.99, 'subsample': <function onehundred at 0x2aae845068c0>, 'unknown_fraction': 0.3, 'scoring_function': <function trident at 0x2aae84506c20>, 'sort': True, 'remove_fragmentary_entries': False, 'fragmentary_threshold': 0.85, 'min_num_entries': 4, 'maas': None, 'remove_only_gaps_entries': False, 'mutation_rates': False, 'force_nucleotides': True, 'convert_N2gap': False, 'input_folder': '/PATH/TO/MAGS/', 'data_folder': '/PATH/TO/MAGS/phylophlan_out/tmp', 'databases_folder': '/home/phylophlan', 'submat_folder': '/home/.local/lib/python3.10/site-packages/phylophlan/phylophlan_substitution_matrices/', 'submod_folder': '/home/.local/lib/python3.10/site-packages/phylophlan/phylophlan_substitution_models/', 'configs_folder': 'phylophlan_configs/', 'output_folder': '', 'genome_extension': '.fasta', 'proteome_extension': '.faa', 'update': False, 'verbose': True}

Loading configuration file "phylophlan_configs/custom_config_aa.cfg"
Checking configuration file
Checking "/home/bin/diamond"
Checking "/home/bin/muscle"
Checking "/home/bin/trimal"
Checking "/home/bin/FastTreeMP"
Checking "/home/bin/raxmlHPC-PTHREADS-SSE3"
[e] database format ("/home/phylophlan/phylophlan/phylophlan.faa", "/home/phylophlan/phylophlan/phylophlan.faa.bz2", or "/home/phylophlan/phylophlan") not recognize

```

---

<div class="post-metadata">

**Author:** ![f.asnicar](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.biobakery.org/f.asnicar/32/196_2.png) [@f.asnicar](https://forum.biobakery.org/u/f.asnicar)\
**Post date:** [February 10, 2024, 12:35pm UTC](https://forum.biobakery.org/t/phylophlan-using-a-manually-downloaded-database/1979/19 "2024-02-10T12:35:05Z")

</div>

Hi @Keaton_Stagaman1, can you post the content of the database folder `/home/phylophlan`?  
Also, when you said that you

> downloaded the phylophlan database manually

Can you be more precise on the process? If you did the download “manually”, I would probably suggest running PhyloPhlAn so that it does the download, uncompression, etc. from a machine with internet access, then kill the process once the database setup is done and re-submit the jobs to the cluster.

Thanks,  
Francesco