You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am working on Gene Regulatory analysis using SCENIC as it has an option to make custom database. Since you and your other colleagues have made the software, I was hoping if you could help me in creating the database for killifish as well. For Killifish we have Ensemble genome (https://www.ensembl.org/Nothobranchius_furzeri/Info/Index ) and Transcription factor list (from the TFDB database) but we do not have any Motif information. Can you let me know if this information is sufficient to proceed? I had a look already into the codes on GitHub, but I still wanted to ask and confirm and seems a bit tough myself to proceed so if you could help me or let me know how it’s done, would be a great help in my research and to the killifish community as well!
For the motif to TF table you could take the fly/mouse/human motif to TF tables we provide and match them to kilifish homologs
(change the gene_name column with the kilifish homologs).
For your gene regions you can get 10kb upstream and downstream of each TSS (so 20kb regions) and/or another set of 500bp upstream and 100 bp downstream of the TSS
for another database (depending on the species you might need to tweak those values a bit e.g 5kb up/down might work better for yours species).
You can use pycistopic tss to get some of the needed files relatively easily:
# Get Ensembl gene annotation name for Killifish.
$ pycistopic tss gene_annotation_list -f killifish
nfurzeri_gene_ensembl Turquoise killifish genes (Nfu_20140520)
$ Get TSS for each gene from Ensembl.
$ pycistopic tss get_tss -n nfurzeri_gene_ensembl -o nfurzeri_gene_ensembl.tss.bed
- Get TSS annotation from Ensembl BioMart with the following settings:
- biomart_name: "nfurzeri_gene_ensembl"
- biomart_host: "http://www.ensembl.org"
- transcript_type: ['protein_coding']
- use_cache: True
- Writing TSS annotation BED file to "nfurzeri_gene_ensembl.tss.bed".
# Sort BED file with TSS.
$ sort -k 1,1 -k2,2n -k3,3n nfurzeri_gene_ensembl.tss.bed > nfurzeri_gene_ensembl.tss.sorted.bed
# Get chromosome name to size mapping.# For most standard species the following would work, but not for Killifish apparently.# Get killifish NCBI accession ID.
$ pycistopic tss get_ncbi_acc -s killifish
accession assembly_name
GCF_027789165.1 UI_Nfuz_MZM_1.0
GCF_001465895.1 Nfu_20140520
$ pycistopic tss get_ncbi_chrom_sizes_and_alias_mapping --ncbi GCF_027789165.1 --chrom-sizes-alias nfurzeri.chrom_sizes_and_alias.tsv
Traceback (most recent call last):
File "/home/luna.kuleuven.be/u0079808/software/anaconda3/envs/pycistopic_3.11/bin/pycistopic", line 8, in<module>sys.exit(main())
^^^^^^
File "/home/luna.kuleuven.be/u0079808/PycharmProjects/pycisTopic/src/pycisTopic/cli/pycistopic.py", line 26, in main
args.func(args)
File "/home/luna.kuleuven.be/u0079808/PycharmProjects/pycisTopic/src/pycisTopic/cli/subcommand/tss.py", line 491, in run_tss_get_ncbi_chrom_sizes_and_alias_mapping
get_chrom_sizes_and_alias_mapping_from_ncbi(
File "/home/luna.kuleuven.be/u0079808/PycharmProjects/pycisTopic/src/pycisTopic/cli/subcommand/tss.py", line 398, in get_chrom_sizes_and_alias_mapping_from_ncbi
ga.get_chrom_sizes_and_alias_mapping_from_ncbi(
File "/home/luna.kuleuven.be/u0079808/PycharmProjects/pycisTopic/src/pycisTopic/gene_annotation.py", line 475, in get_chrom_sizes_and_alias_mapping_from_ncbi
chrom_sizes_and_alias_df_pl = pl.from_dicts(reports).select(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/luna.kuleuven.be/u0079808/software/anaconda3/envs/pycistopic_3.11/lib/python3.11/site-packages/polars/dataframe/frame.py", line 8873, inselectreturnself.lazy().select(*exprs, **named_exprs).collect(_eager=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/luna.kuleuven.be/u0079808/software/anaconda3/envs/pycistopic_3.11/lib/python3.11/site-packages/polars/lazyframe/frame.py", line 2034, in collect
return wrap_df(ldf.collect(callback))
^^^^^^^^^^^^^^^^^^^^^
polars.exceptions.ColumnNotFoundError: ucsc_style_name
# For Killifish you can use get_chromosome_sizes: https://gist.github.com/andrewyatz/a3687b573364f65904e2
./get_chromosome_sizes.py 'nothobranchius_furzeri'> nothobranchius_furzeri.chrom_sizes.tsv
# Create BED file with regions 20kb up and downstream of TSS.
$ bedtools slop -l 20000 -r 20000 -s -i nfurzeri_gene_ensembl.tss.sorted.bed -g nothobranchius_furzeri.chrom_sizes.tsv > nfurzeri_gene_ensembl.tss.sorted.20kb_up_and_down.bed
# Create BED file with regions 500bp upstream and 100bp downstream of TSS.
$ bedtools slop -l 500 -r 100 -s -i nfurzeri_gene_ensembl.tss.sorted.bed -g nothobranchius_furzeri.chrom_sizes.tsv > nfurzeri_gene_ensembl.tss.sorted.500bp_up_and_100bp_down.bed
Hi, thank you! But where is this BED files needed? To form the create_cisTarget_databases I need fasta file and motif file right? The tutorial is quite confusing. Can you please just tell what to do after this?
Hi, So I tried the above command but I ran into 2 errors:
1. Error: Non-zero exit status for: '/staging/leuven/stg_00056/Juhi/Juhi/scenicplus/scenicplus/data/create_cisTarget_databases/cluster-buster/cbust -f 4 -c 0.0 -r 10000 -b 0 -t 1 /staging/leuven/stg_00056/Juhi/Juhi/scenicplus/scenicplus/data/v10nr_clust_public/singletons/metacluster_142.4.cb /staging/leuven/stg_00056/Juhi/Juhi/scenicplus/scenicplus/data/killifish_gene_regions_20kb_up_and_down.fa'
2. Error: Only 10072 out of 10249 motifs were scored successfully.
Kindly let me know how to process this successfully. After running the whole command, these were the errors that I encountered. Hence there was no output after running them.
Hi, so there wasn't just the error with this one, but it had more, I just sent the snippet of it. I don't know why this came and why there wasn't any output since it said:
Error: Only 10072 out of 10249 motifs were scored successfully
Activity
ghuls commentedon Sep 18, 2024
For the motif to TF table you could take the fly/mouse/human motif to TF tables we provide and match them to kilifish homologs
(change the gene_name column with the kilifish homologs).
For your gene regions you can get 10kb upstream and downstream of each TSS (so 20kb regions) and/or another set of 500bp upstream and 100 bp downstream of the TSS
for another database (depending on the species you might need to tweak those values a bit e.g 5kb up/down might work better for yours species).
You can use
pycistopic tssto get some of the needed files relatively easily:ghuls commentedon Sep 18, 2024
Files from previous post attached:
nfurzeri_gene_ensembl.tss.sorted.zip
juhi-ku commentedon Sep 24, 2024
Hi, thank you! But where is this BED files needed? To form the create_cisTarget_databases I need fasta file and motif file right? The tutorial is quite confusing. Can you please just tell what to do after this?
ghuls commentedon Sep 30, 2024
You need the BED file to create a FASTA file with the sequences for your gene regions.
Cluster-Buster motif files from our public collection can be found here:
https://resources.aertslab.org/cistarget/motif_collections/v10nr_clust_public/v10nr_clust_public.zip
juhi-ku commentedon Nov 15, 2024
Hi, So I tried the above command but I ran into 2 errors:
Kindly let me know how to process this successfully. After running the whole command, these were the errors that I encountered. Hence there was no output after running them.
ghuls commentedon Nov 18, 2024
What is the output (error) if you run:
/staging/leuven/stg_00056/Juhi/Juhi/scenicplus/scenicplus/data/create_cisTarget_databases/cluster-buster/cbust \ -f 4 \ -c 0.0 \ -r 10000 \ -b 0 \ -t 1 \ /staging/leuven/stg_00056/Juhi/Juhi/scenicplus/scenicplus/data/v10nr_clust_public/singletons/metacluster_142.4.cb \ /staging/leuven/stg_00056/Juhi/Juhi/scenicplus/scenicplus/data/killifish_gene_regions_20kb_up_and_down.fa \ > metacluster_142.4.cbust.tsvjuhi-ku commentedon Nov 18, 2024
Hi, so this is the script that I ran:
ghuls commentedon Nov 19, 2024
But was is the output and error message from this command (as that one seemed to have failed)?
juhi-ku commentedon Nov 19, 2024
Hi, so there wasn't just the error with this one, but it had more, I just sent the snippet of it. I don't know why this came and why there wasn't any output since it said:
ghuls commentedon Nov 20, 2024
Three reason I could think of why those failed without knowing the error message (as you didn't post the output of the command I asked):