Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

I am using a new bioinformatics tool called Giggle and I have installed the python wrapper on my system. Even though the scenario is quite specific, I think the problem is quite general. This function:

index = Giggle.create("index", "HMEC_hg19_BroadHMM_ALL.bed")
should create an index based on several (or in this case one) .bed file.
The bed files look like this: 
chr1    10000   10600   15_Repetitive/CNV   0   .   10000   10600   245,245,245
chr1    10600   11137   13_Heterochrom/lo   0   .   10600   11137   245,245,245
chr1    11137   11737   8_Insulator 0   .   11137   11737   10,190,254
chr1    11737   11937   11_Weak_Txn 0   .   11737   11937   153,255,102
chr1    11937   12137   7_Weak_Enhancer 0   .   11937   12137   255,252,4
chr1    12137   14537   11_Weak_Txn 0   .   12137   14537   153,255,102
chr1    14537   20337   10_Txn_Elongation   0   .   14537   20337   0,176,80
It is basically a large tab delimited file containing genomic intervals and their corresponding chromosome. When running the above command I get the following error:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "giggle/giggle.pyx", line 25, in giggle.giggle.Giggle.create
TypeError: expected bytes, str found
I have no clue why this is happening and I have tried converting the files to other types of encoding but nothing worked. The code snippet to which the error refers is as follows:
def create(self, char *path, char *glob):
    giggle_bulk_insert(to_bytes(glob), to_bytes(path), 1)
    return Giggle(path)
I am using Python 3.6 on a Linux subsystem for windows 10. 
The problem is that in python 3 strings are represented as unicode strings, not byte strings as it was the case in python 2. When you install giggle and run your code using python 2 everything works fine. But you can do:
index = Giggle.create("index".encode('utf-8'), "HMEC_hg19_BroadHMM_ALL.bed".encode('utf-8'))
or alternatively 
index = Giggle.create(b"index", b"HMEC_hg19_BroadHMM_ALL.bed")
to have explicit byte strings. It worked for me, up to the point that giggle complains about the .bed file being incorrectly formatted (I probably messed up the format when copying)
Update:
There is another issue that comes up when calling it like described above:
  
File type not supported 'HMEC_hg19_BroadHMM_ALL.bed'
Which is caused by the underlying lib giggle only accepting .bed.gz files, which can be seen in python-giggle/lib/giggle/src/file_read.c:
if ( (strlen(i->file_name) > 7) &&
    strcmp(".bed.gz", file_name + strlen(i->file_name) - 7) == 0) {
    i->type = BED;
So I am assuming that the Readme at the python-giggle site is not correct in claiming that you can call it with .bed files.
I tested it with one of the files provided in python-giggle\lib\giggle\test\data and it ran without an error
                I am both suprised and amazed you got it working so quickly. When I do the same in a python2.7 env I get the following error: File type not supported 'GM12878_hg19_BroadHMM_ALL.bed'. Segmentation fault (core dumped). When I try either the .encode method or the b method in python3.6 I get the same error and when running tests in a jupyter notebook, my kernel instantly dies running the create index function. Since you got it working so quickly, could je help me on my way?
– Matthijs van Kesteren
                Feb 13, 2018 at 13:45
                I got the same error but assumed it was due to me messing up the format (I have no idea what it is supposed to look like). But I found the solution I think, which is quite odd thoug
– FlyingTeller
                Feb 13, 2018 at 14:07
                You have a very sharp eye. However, I am still not managing to run it on the test data without errors: In [1]: from giggle import Giggle  In [2]: index = Giggle.create(b"new_index", b"1k.sort.bed.gz") python: offset_index_store: Could not truncate.: Invalid argument Which file did you exactly use?
– Matthijs van Kesteren
                Feb 13, 2018 at 14:34
                You mean your own data or some from python-giggle\lib\giggle\test\data? What error are you getting?
– FlyingTeller
                Feb 13, 2018 at 14:36
Cython can only accept bytes objects in Python 3, str in Python 2, to convert to a char array automatically.
Either pass in bytes objects when you call the method (encoding your str objects first), or alter that method signature to accept str unicode strings. See Accepting strings from Python code in the Cython tutorial. 
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.