By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement . We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hi community,

I'm porting the OpenThread stack to RIOT. As I wrote in devel, I'm having a hard fault when I link RIOT with the core of OT. This error is happening with all ARM boards. I have been using a SAMR21-xpro.

RIOT is crashing in this line [ 1 ], function libc_init_array(). Here's the content of hardfault:

2016-06-17 13:40:42,716 - INFO # Context before hardfault:
2016-06-17 13:40:42,718 - INFO #    r0: 0x00000000
2016-06-17 13:40:42,719 - INFO #    r1: 0x00000655
2016-06-17 13:40:42,721 - INFO #    r2: 0x00000000
2016-06-17 13:40:42,722 - INFO #    r3: 0x000058c4
2016-06-17 13:40:42,724 - INFO #   r12: 0x00000000
2016-06-17 13:40:42,726 - INFO #    lr: 0x00021ea7
2016-06-17 13:40:42,727 - INFO #    pc: 0x000058c4
2016-06-17 13:40:42,729 - INFO #   psr: 0x40000000
2016-06-17 13:40:42,729 - INFO # 
2016-06-17 13:40:42,729 - INFO # Misc
2016-06-17 13:40:42,731 - INFO # EXC_RET: 0xfffffff9
2016-06-17 13:40:42,735 - INFO # Attempting to reconstruct state for debugging...
2016-06-17 13:40:42,736 - INFO # In GDB:
2016-06-17 13:40:42,737 - INFO #   set $pc=0x58c4
2016-06-17 13:40:42,738 - INFO #   frame 0
2016-06-17 13:40:42,739 - INFO #   bt

To reproduce the error, try the example in [2](it's still in development phase, sorry for the mess):

  • Run make BOARD=samr21-xpro all. It will through an error due to unfinished Makefile
  • Go to bin/pkg/samr21-xpro/openthread
  • Run ./bootstrap
  • run ./configure --enable-cli --host=arm-none-eabi
  • Run make BOARD=samr21-xpro clean all flash term
  • I have been searching on internet and it seems I'm not the only one who has problem with libc_init_array and C++. Might be something like [3]?

    The example is running on RIOT's native, so I think is not a problem of the port itself. Also, here in [4] is the .map file.

    Hi, trying to reproduce the error. I checked out your RIOT fork and followed your steps, but I am running into this error:

    /Users/noir/Desktop/RIOT/pkg/openthread/contrib/openthread.c:37:8: error: unknown type name 'netdev2_tap_t'
     extern netdev2_tap_t netdev2_tap;
    

    Am I missing something?

    Hi @josephnoir
    Sorry, it was my fault. When I added netdev2_tap interface I forgot to add #ifdef MODULE_NETDEV2_TAP to the extern declaration.

    Just fixed it. Try now.
    Cheers

    @jia200x How did you figure our where it crashes? My debugger (well, make BOARD=samr21-xpr debug) isn't working, it just hangs. And when I objdump the elf file and look at the address of $pc, ...

    2016-06-22 13:48:26,172 - INFO # Context before hardfault:
    2016-06-22 13:48:26,173 - INFO #    r0: 0x00000000
    2016-06-22 13:48:26,175 - INFO #    r1: 0x00000000
    2016-06-22 13:48:26,176 - INFO #    r2: 0xe000e100
    2016-06-22 13:48:26,178 - INFO #    r3: 0x00005f10
    2016-06-22 13:48:26,179 - INFO #   r12: 0x00000003
    2016-06-22 13:48:26,181 - INFO #    lr: 0x00028fdf
    2016-06-22 13:48:26,183 - INFO #    pc: 0x00005f10
    2016-06-22 13:48:26,184 - INFO #   psr: 0x40000000
    2016-06-22 13:48:26,184 - INFO # 
    2016-06-22 13:48:26,185 - INFO # Misc
    2016-06-22 13:48:26,186 - INFO # EXC_RET: 0xfffffff9
    2016-06-22 13:48:26,191 - INFO # Attempting to reconstruct state for debugging...
    2016-06-22 13:48:26,191 - INFO # In GDB:
    2016-06-22 13:48:26,193 - INFO #   set $pc=0x5f10
    2016-06-22 13:48:26,194 - INFO #   frame 0
    2016-06-22 13:48:26,194 - INFO #   bt
    

    ... this is what I see:

    $ arm-none-eabi-objdump -d bin/samr21-xpro/openthread_example.elf
    [...]
    00005f10 <_GLOBAL__sub_I__ZN6Thread3Ip63Ip610NewMessageEt>:
        5f10:   e92d4010    push    {r4, lr}
        5f14:   e59f0008    ldr r0, [pc, #8]    ; 5f24 <_GLOBAL__sub_I__ZN6Thread3Ip63Ip610NewMessageEt+0x14>
        5f18:   eb00023d    bl  6814 <_ZN6Thread3Ip63MplC1Ev>
        5f1c:   e8bd4010    pop {r4, lr}
        5f20:   e12fff1e    bx  lr
        5f24:   20004480    .word   0x20004480
    Try debugging with make BOARD=samr21-xpro debug, and put a breakpoint in file vectors_cortexm.c (e.g line 97)

    Cheers

    Tried that with lines 73 and 95. This is all I see:

    $ make BOARD=samr21-xpro debug
    /Users/noir/Desktop/RIOT/dist/tools/openocd/openocd.sh debug
    ### Starting Debugging ###
    Reading symbols from /Users/noir/Desktop/RIOT/examples/openthread_example/bin/samr21-xpro/openthread_example.elf...Open On-Chip Debugger 0.9.0 (2015-11-15-05:39)
    Licensed under GNU GPL v2
    For bug reports, read
        http://openocd.org/doc/doxygen/bugs.html
    done.
    Remote debugging using :3333
    hard_fault_handler (sp=0x2 <interrupt_vector+2>, corrupted=<optimized out>, exc_return=0, 
        r4_to_r11_stack=0x1b4 <sched_run>) at /Users/noir/Desktop/RIOT/cpu/cortexm_common/vectors_cortexm.c:298
    298         __BKPT(1);
    (gdb) break vectors_cortexm.c:97
    Breakpoint 1 at 0x498: file /Users/noir/Desktop/RIOT/cpu/cortexm_common/vectors_cortexm.c, line 97.
    (gdb) run
    The program being debugged has been started already.
    Start it from the beginning? (y or n) y
    Starting program: /Users/noir/Desktop/RIOT/examples/openthread_example/bin/samr21-xpro/openthread_example.elf 
    Note: automatically using hardware breakpoints for read-only addresses.
    

    Or does it take more than 5 minutes to advance? (I am on OS X, btw.)

    I'm on OS X too, you should see the error instantly.

    Try putting a breakpoint in the libc_init_array function or in another function (you might try to put a breakpoint in the main_trampoline). Maybe for you is crashing in another part.

    Cheers

    Yep. If I am not mistaken it allows static initialization for classes in C++. When first trying to get static initialization on the stm32f4discovery, we called the init_array functions by hand in a startup file.

    When I append the implementation of __libc_init_array to the syscalls.c file I can step into it and it crashes in the second loop. The function should morally be provided by the newlib, right?

    However, I am still wondering if why we don't call the function from the newlib. A small test that uses static initialization works regardless.

    The objdump shows the function anyway. First one is from the newlib.

    00028f04 <__libc_init_array>:
       28f04:   4b0e        ldr r3, [pc, #56]   ; (28f40 <__libc_init_array+0x3c>)
       28f06:   b570        push    {r4, r5, r6, lr}
       28f08:   2500        movs    r5, #0
       28f0a:   001e        movs    r6, r3
       28f0c:   4c0d        ldr r4, [pc, #52]   ; (28f44 <__libc_init_array+0x40>)
       28f0e:   1ae4        subs    r4, r4, r3
       28f10:   10a4        asrs    r4, r4, #2
       28f12:   42a5        cmp r5, r4
       28f14:   d004        beq.n   28f20 <__libc_init_array+0x1c>
       28f16:   00ab        lsls    r3, r5, #2
       28f18:   58f3        ldr r3, [r6, r3]
       28f1a:   4798        blx r3
       28f1c:   3501        adds    r5, #1
       28f1e:   e7f8        b.n 28f12 <__libc_init_array+0xe>
       28f20:   f7d7 f8c6   bl  b0 <_init>
       28f24:   4b08        ldr r3, [pc, #32]   ; (28f48 <__libc_init_array+0x44>)
       28f26:   2500        movs    r5, #0
       28f28:   001e        movs    r6, r3
       28f2a:   4c08        ldr r4, [pc, #32]   ; (28f4c <__libc_init_array+0x48>)
       28f2c:   1ae4        subs    r4, r4, r3
       28f2e:   10a4        asrs    r4, r4, #2
       28f30:   42a5        cmp r5, r4
       28f32:   d004        beq.n   28f3e <__libc_init_array+0x3a>
       28f34:   00ab        lsls    r3, r5, #2
       28f36:   58f3        ldr r3, [r6, r3]
       28f38:   4798        blx r3
       28f3a:   3501        adds    r5, #1
       28f3c:   e7f8        b.n 28f30 <__libc_init_array+0x2c>
       28f3e:   bd70        pop {r4, r5, r6, pc}
       28f40:   0002f594    .word   0x0002f594
       28f44:   0002f594    .word   0x0002f594
       28f48:   0002f594    .word   0x0002f594
       28f4c:   0002f59c    .word   0x0002f59c
    

    And this one is the self implemented one.

    00000168 <__libc_init_array>:
         168:   4b0e        ldr r3, [pc, #56]   ; (1a4 <__libc_init_array+0x3c>)
         16a:   b570        push    {r4, r5, r6, lr}
         16c:   2500        movs    r5, #0
         16e:   001e        movs    r6, r3
         170:   4c0d        ldr r4, [pc, #52]   ; (1a8 <__libc_init_array+0x40>)
         172:   1ae4        subs    r4, r4, r3
         174:   10a4        asrs    r4, r4, #2
         176:   42a5        cmp r5, r4
         178:   d004        beq.n   184 <__libc_init_array+0x1c>
         17a:   00ab        lsls    r3, r5, #2
         17c:   58f3        ldr r3, [r6, r3]
         17e:   4798        blx r3
         180:   3501        adds    r5, #1
         182:   e7f8        b.n 176 <__libc_init_array+0xe>
         184:   f000 fa90   bl  6a8 <uart_stdio_init>
         188:   4b08        ldr r3, [pc, #32]   ; (1ac <__libc_init_array+0x44>)
         18a:   2500        movs    r5, #0
         18c:   001e        movs    r6, r3
         18e:   4c08        ldr r4, [pc, #32]   ; (1b0 <__libc_init_array+0x48>)
         190:   1ae4        subs    r4, r4, r3
         192:   10a4        asrs    r4, r4, #2
         194:   42a5        cmp r5, r4
         196:   d004        beq.n   1a2 <__libc_init_array+0x3a>
         198:   00ab        lsls    r3, r5, #2
         19a:   58f3        ldr r3, [r6, r3]
         19c:   4798        blx r3
         19e:   3501        adds    r5, #1
         1a0:   e7f8        b.n 194 <__libc_init_array+0x2c>
         1a2:   bd70        pop {r4, r5, r6, pc}
         1a4:   0002f594    .word   0x0002f594
         1a8:   0002f594    .word   0x0002f594
         1ac:   0002f594    .word   0x0002f594
         1b0:   0002f59c    .word   0x0002f59c
    

    Difference seems to be the inline call to uart_stdio_init vs the generic _init

    As I was reading in (1), the __init_array_start is provided by the runtime linker. I'm checking if the pointer to that function is valid.

    Maybe @haukepetersen or @kaspar030 has something to say about this?

    Cheers.

    The __init_array_start marker can be found in the linker file. Not sure where it points to at runtime.

    This is the code we used to hack in static initialization in native earlier. Was never part of the master branch, just for testing. It doesn't call anything from the array until it finds the function name that called it (the startup function). iirc, @dangnhat helped me with that.

    Doing a similar thing in the vectors_cortexm.c results in no calls during the loop and I get to main.

    #if MODULE_NEWLIB
        /* initialize std-c library (this must be done after board_init) */
        // extern void __libc_init_array(void);
        // __libc_init_array();
        /* manually call other contructors in __init_array which haven't been called */
        #include "uart_stdio.h"
        uart_stdio_init();
        typedef void (*func_ptr)(void);
        extern func_ptr __init_array_start[];
        extern func_ptr __init_array_end[];
        int size = __init_array_end - __init_array_start;
        int i, flag = 0;
        for (i = 0; i < size; i++) {
            if (__init_array_start[i] == reset_handler_default) {
                flag = 1;
                continue;
            if (flag == 1){
                (__init_array_start[i])();
    #endif

    Doesn't sound like the right way to fix it though. Are we somehow adding something to the init array that shouldn't be there or does the initialization break something else?

    Good snippet!
    Yes, I'm not sure what's going on there but at least I think I can use that for now to continue testing OpenThread.
    Cheers

    How many calls does the __init_array loop skip before it reaches the reset_handler_default function? And, does is make a difference if you add the loop to call the __preinit_array functions?

      size = __preinit_array_end - __preinit_array_start;
      for (i = 0; i < size; i++)
        __preinit_array_start[i] ();

    Actually never reaches the reset_handler_default function. So just skips
    the init arrays. At some point this makes the program crash.

    It makes no difference when I add or remove the __preinit part.

    2016-06-23 16:00 GMT+02:00 Joseph Noir notifications@github.com:

    How many calls does the __init_array loop skip before it reaches the
    reset_handler_default function? And, does is make a difference if you add
    the loop to call the __preinit_array functions?

    size = __preinit_array_end - __preinit_array_start;
    for (i = 0; i < size; i++)
    __preinit_array_starti;

    You are receiving this because you were mentioned.
    Reply to this email directly, view it on GitHub
    #5563 (comment), or mute
    the thread
    https://github.com/notifications/unsubscribe/ABM8SBxYSUjvsyGMiKLOjzNF9cZ2h3Ulks5qOpFkgaJpZM4I63gr

    I discovered something and I'm not sure if it's the expected behavior.

    (gdb) p __preinit_array_start
    $18 = 0x2c7ec <__preinit_array_start>
    (gdb) p __preinit_array_end
    $19 = 0x2c7ec <__preinit_array_start>
    (gdb) p __init_array_start
    $20 = 0x2c7ec <__preinit_array_start>
    (gdb) p __init_array_end
    $21 = 0x2c7f4
    (gdb) p __init_array_end-__init_array_start
    $22 = 2
    

    As you can see, __preinit_array_start points to the same place as
    __init_array_start. __preinit_array_start is the same as
    __preinit_array_end (so, __preinit_array_start functions are not being
    executed).

    2016-06-23 20:12 GMT+02:00 Jose Alamos jialamos@uc.cl:

    Actually never reaches the reset_handler_default function. So just skips
    the init arrays. At some point this makes the program crash.

    It makes no difference when I add or remove the __preinit part.

    2016-06-23 16:00 GMT+02:00 Joseph Noir notifications@github.com:

    How many calls does the __init_array loop skip before it reaches the
    reset_handler_default function? And, does is make a difference if you
    add the loop to call the __preinit_array functions?

    size = __preinit_array_end - __preinit_array_start;
    for (i = 0; i < size; i++)
    __preinit_array_starti;

    You are receiving this because you were mentioned.
    Reply to this email directly, view it on GitHub
    #5563 (comment), or mute
    the thread
    https://github.com/notifications/unsubscribe/ABM8SBxYSUjvsyGMiKLOjzNF9cZ2h3Ulks5qOpFkgaJpZM4I63gr

    I have compiled the rest of the RIOT examples and some random codes, and I couldn't find a code that has __preinit and __init functions. This is what objdump tells about the init and preinit functions in the OT port:

    0002c7f4 g       .text  00000000 __fini_array_end
    0002c7f4 g       .text  00000000 __fini_array_start
    0002c7f4 g       .text  00000000 __init_array_end
    0002c7ec g       .text  00000000 __preinit_array_end
    0002c7ec g       .text  00000000 __init_array_start
    0002c7ec g       .text  00000000 __preinit_array_start
    
    0002c7ec <__init_array_start>:
       2c7ec:   000066cc    .word   0x000066cc
       2c7f0:   0001b608    .word   0x0001b608
    

    For example, the RIOT and CPP example shows this:

    0000c130 g       .text  00000000 __fini_array_end
    0000c130 g       .text  00000000 __fini_array_start
    0000c130 g       .text  00000000 __init_array_end
    0000c130 g       .text  00000000 __preinit_array_end
    0000c130 g       .text  00000000 __init_array_start
    0000c130 g       .text  00000000 __preinit_array_start
    

    So, basically there aren't functions in the init and preinit there. About my previous comment, these functions are pointing to the same place due to compiler optimizations.

    Not sure what the exact purpose or __preinit_array is, but it may very well be empty. The code I posted above simply skips all calls in the __init_array. However, calling any of the functions leads to a hard fault. It is still curious that I do get to main that way ...

    @gebart, @dangnhat do you have a bit of time to look at the problem or a helpful comment?

    Yes, I got the same too, but with a lot of uncompleted features. It seems these not called init functions are making the main crash at random parts.

    So, it goes into the main but anyway fails. I'm not sure why this is the only code that has calls to the __preinit and __init functions.

    The second constructor call of __init_array maybe for the sCommand array in RIOT/examples/openthread_example/bin/pkg/samr21-xpro/openthread/src/cli/cli.cpp line 49. Not sure about the first one though. This is the address in the array:

    00005eac <_GLOBAL__sub_I__ZN6Thread3Ip63Ip610NewMessageEt>:
        5eac:   e92d4010    push    {r4, lr}
        5eb0:   e59f0008    ldr r0, [pc, #8]    ; 5ec0 <_GLOBAL__sub_I__ZN6Thread3Ip63Ip610NewMessageEt+0x14>
        5eb4:   eb00023d    bl  67b0 <Thread::Ip6::Mpl::Mpl()>
        5eb8:   e8bd4010    pop {r4, lr}
        5ebc:   e12fff1e    bx  lr
        5ec0:   20004558    .word   0x20004558

    Yes, that's exactly where is crashing. It means it's something with OpenThread constructors.
    Actually, it's crashing with a ands r0, r1 instruction. I don't know what's going on, these registers are for general purpose.

    guys, just want to add some information which might be helpful for you guys:
    (short version: the code snippet is not necessary for ARM platform, don't use it; openthread might have added some initialization routines in init_arrays and make our program crashes => IMHO, we should test it without RIOT first :-) )

  • First, the code snippet above only works in native as a workaround for RIOT current implementation where RIOT's kernel_init is implemented as a constructor (i.e. C initialization routine [1]) and RIOT will miss some user's initialization routines (defined after kernel_init in the code) in init_array . The names for these initialization arrays depend on compilers so they could be different between native implementations and ARM platforms. On the other hand, RIOT implementation on ARM platforms is a little bit different from native because we could access to startup code on ARM (while we couldn't in native).Therefore, our program will be fully initialized before calling kernel_init (without the code snippet).
  • Second, __pre_init_array __init_array [2] will be filled with initialization routines by linker during link time and user code could define some additional routines to be filled in too. Therefore, IMHO, openthread port for ARM might has added some initialization routines which make our program crashes => we should test openthread port alone (without RIOT) on ARM to see whether it crashes or not.
  • I will try @jia200x's example on my stm32 device and will let you guys know if I find anything useful. :-)

    Cheers,
    Nhat.

    [1] https://gcc.gnu.org/onlinedocs/gccint/Initialization.html
    [2] http://dbp-consulting.com/tutorials/debugging/linuxProgramStartup.html

    Usually, due to memory latency, and interrupt latency, the instruction
    pointed to at a hard fault is usually one or two instructions after the
    actual memory access violation. You should be looking for a ldr or str
    instruction close to your ands.

    But yeah, I agree with @josephnoir that the issue is likely in the
    constructors. I'm guessing that the constructor expects some other piece to
    have already been initialized, which hasn't been.
    I'm replying from my phone so I can't really look into anything now, but is
    everything from .data and .bss initialized when the constructor is started?
    Is the crashing constructor accessing any peripherals or kernel stuff that
    still hasn't been initialized? For example printf/puts for debugging or any
    spi or UART, or any riot ipc/messages for communicating with the network
    stack.
    Does anyone in this thread have the time to hack together a tiny C++ static
    constructor test as a regression test to include in RIOT?
    I'm thinking that something like getting the CPU ID from inside the
    constructor should prevent it from being optimized away during compilation.
    It will not be possible to use printf until after the stdio initialization
    has been run during board init so it won't be possible to print stuff
    during constructor calls on most platforms.
    Den 24 jun 2016 18:21 skrev "jia200x" notifications@github.com:

    Yes, that's exactly where is crashing. It means it's something with
    OpenThread constructors.
    Actually, it's crashing with a ands r0, r1 instruction. I don't know
    what's going on, these registers are for general purpose.

    You are receiving this because you were mentioned.
    Reply to this email directly, view it on GitHub
    #5563 (comment), or mute
    the thread
    https://github.com/notifications/unsubscribe/AATYQjwPVWgDs4Iax8lmOgF4I0f6fdk6ks5qPAQNgaJpZM4I63gr

    Thanks for the input!

    @dangnhat Isn't OpenThread made for these devices? Still, validating that it actually runs on its own would help to exclude some sources for errors.

    The names for these initialization arrays depend on compilers so they could be different between native implementations and ARM platforms.

    I wasn't sure about the code we used for native, but also looked at the init.c of the newlib and used their implementation. Somewhere above is the objdump of both (using the function form newlib and implementing it ourselves). Looks pretty similar. I think the __preinit_array is empty in both cases and the __init_array has two entries.

    I will try @jia200x's example on my stm32 device and will let you guys know if I find anything useful. :-)

    Awesome. Hope you find something ^^

    @gebart How can we test if the sections are initialized correctly?

    Usually, due to memory latency, and interrupt latency, the instruction
    pointed to at a hard fault is usually one or two instructions after the
    actual memory access violation. You should be looking for a ldr or str
    instruction close to your ands.

    That's good to know. I guess it would be helpful to know what constructor is called and have a look at the code ...

    Does anyone in this thread have the time to hack together a tiny C++ static
    constructor test as a regression test to include in RIOT?

    Sounds like a good idea, is something like this enough?

    Thank you for all the information!
    With the following asm code:

       25d10:       00a3            lsls    r3, r4, #2
       25d12:       58f3            ldr     r3, [r6, r3]
       25d14:       4798            blx     r3
    

    The r3 register is 0x6374. The disassembly show this in that address:

     6374:  e92d4008    push    {r3, lr}
        6378:   e59f0008    ldr r0, [pc, #8]    ; 6388 <_GLOBAL__sub_I__ZN6Thread3Ip63Ip610NewMessageEt+0x14>
        637c:   eb000249    bl  6ca8 <_ZN6Thread3Ip63MplC1Ev>
        6380:   e8bd4008    pop {r3, lr}
        6384:   e12fff1e    bx  lr
        6388:   200068e0    .word   0x200068e0
    

    But GDB is showing this:

    0x6374 <_GLOBAL__sub_I__ZN6Thread3Ip63Ip610NewMessageEt()>      ands   r0, r1                                                                                              
    0x6376 <_GLOBAL__sub_I__ZN6Thread3Ip63Ip610NewMessageEt()+2>    stmdb  sp!, {r3}                                                                                           
    0x637a <_GLOBAL__sub_I__ZN6Thread3Ip63Ip610NewMessageEt()+6>    b.n    0x5ebc <Thread::Ip6::HandlePayload(Thread::Message&, Thread::Ip6::MessageInfo&, unsigned char)+8>   │
    0x637c <_GLOBAL__sub_I__ZN6Thread3Ip63Ip610NewMessageEt()+8>    lsls   r1, r1, #9                                                                                          
    0x637e <_GLOBAL__sub_I__ZN6Thread3Ip63Ip610NewMessageEt()+10>   add.w  r0, r0, r8, lsl #16          
    

    I couldn't find some of these lines in the disassembly. When I try to jump to 0x6374 it throws the same hard fault.

    The second __init_array_start pointer is pointing to GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE (Static initializer of sCommands). Again, according to the objdump disassembly file the asm is:

    0001b558 <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE>:
       1b558:   e92d4010    push    {r4, lr}
       1b55c:   e59f4034    ldr r4, [pc, #52]   ; 1b598 <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE+0x40>
       1b560:   e3a02000    mov r2, #0
       1b564:   e2840018    add r0, r4, #24
       1b568:   e59f102c    ldr r1, [pc, #44]   ; 1b59c <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE+0x44>
       1b56c:   ebffe078    bl  13754 <_ZN6Thread3Ip68IcmpEchoC1EPFvPvRNS_7MessageERKNS0_11MessageInfoEES2_>
    

    GDB shows this:

    (gdb) x/10i 0x1b558
       0x1b558 <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()>:   ands    r0, r2
       0x1b55a <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+2>: stmdb   sp!, {r2, r4, r5, lr}
       0x1b55e <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+6>: b.n     0x1b0a0 <Thread::Cli::Interpreter::ProcessPrefixAdd(int, char**)+416>
       0x1b560 <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+8>: movs    r0, #0
       0x1b562 <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+10>:        b.n     0x1bca6 <mbedtls_aes_encrypt+218>
       0x1b564 <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+12>:        movs    r0, r3
       0x1b566 <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+14>:        b.n     0x1ba72 <mbedtls_aes_setkey_dec+46>
       0x1b568 <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+16>:        asrs    r4, r5, #32
       0x1b56a <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+18>:        b.n     0x1b0ac <Thread::Cli::Interpreter::ProcessPrefixAdd(int, char**)+428>
       0x1b56c <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+20>:        b.n     0x1b660 <otCryptoHmacSha256Start+36>
    

    Why is this inconsistency? Other addresses point to the right place according to the objdump disassembly file. This is only happening with the __init_array functions.

    Which inconsistency are you referring to?
    I think gdb gets confused and tries to pretty print whatever you tried to
    print as arm mode machine code. If you want to look at the assembly it's
    better to use the disassemble command in gdb. It would be interesting to
    see what the pointer at 0x1b598 is pointing to since that seem to be the
    address that is causing the trouble.
    Den 27 jun 2016 15:54 skrev "jia200x" notifications@github.com:

    The second __init_array_start pointer is pointing to
    GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE (Static initializer of
    sCommands). Again, according to the objdump disassembly file the asm is:

    0001b558 <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE>:
    1b558: e92d4010 push {r4, lr}
    1b55c: e59f4034 ldr r4, [pc, #52] ; 1b598 <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE+0x40>
    1b560: e3a02000 mov r2, #0
    1b564: e2840018 add r0, r4, #24
    1b568: e59f102c ldr r1, [pc, #44] ; 1b59c <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE+0x44>
    1b56c: ebffe078 bl 13754 <ZN6Thread3Ip68IcmpEchoC1EPFvPvRNS_7MessageERKNS0_11MessageInfoEES2>

    GDB shows this:

    (gdb) x/10i 0x1b558
    0x1b558 <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()>: ands r0, r2
    0x1b55a <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+2>: stmdb sp!, {r2, r4, r5, lr}
    0x1b55e <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+6>: b.n 0x1b0a0 <Thread::Cli::Interpreter::ProcessPrefixAdd(int, char**)+416>
    0x1b560 <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+8>: movs r0, #0
    0x1b562 <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+10>: b.n 0x1bca6 <mbedtls_aes_encrypt+218>
    0x1b564 <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+12>: movs r0, r3
    0x1b566 <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+14>: b.n 0x1ba72 <mbedtls_aes_setkey_dec+46>
    0x1b568 <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+16>: asrs r4, r5, #32
    0x1b56a <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+18>: b.n 0x1b0ac <Thread::Cli::Interpreter::ProcessPrefixAdd(int, char**)+428>
    0x1b56c <_GLOBAL__sub_I__ZN6Thread3Cli11Interpreter9sCommandsE()+20>: b.n 0x1b660 <otCryptoHmacSha256Start+36>

    Why is this inconsistency? Other addresses point to the right place
    according to the objdump disassembly file. This is only happening with the
    __init_array functions.

    You are receiving this because you were mentioned.
    Reply to this email directly, view it on GitHub
    #5563 (comment), or mute
    the thread
    https://github.com/notifications/unsubscribe/AATYQuc4qwgcOf48R1xyv9yylSxcCbzsks5qP9YagaJpZM4I63gr

    I mean, for the same address GDB and objdump show different instructions.

    Thumb instruction set of cortex-m0 looks usually looks like:

    1af4:       **b538**            push    {r3, r4, r5, lr}
        1af6:       **6001**            str     r1, [r0, #0]
        1af8:       **6901**            ldr     r1, [r0, #16]
        1afa:       **6943**            ldr     r3, [r0, #20]
    

    16 bit instructions (sometimes 32, but it depends on the instruction).

    Some of the OpenThread functions are being shown like this in the assembly file:

       1f70:       **e3520000**        cmp     r2, #0
        1f74:       **13510000**        cmpne   r1, #0
        1f78:       **e92d4010**        push    {r4, lr}
        1f7c:       **e1a0c001**        mov     ip, r1
    

    So basically objdump is trying to interpret these 32 bits instructions (when they should be 16 bit). It's an objdump problem?

    The pointer that is causing problems is the first one (0x6374), that points here:

    6374:  e92d4008    push    {r3, lr}
        6378:   e59f0008    ldr r0, [pc, #8]    ; 6388 <_GLOBAL__sub_I__ZN6Thread3Ip63Ip610NewMessageEt+0x14>
        637c:   eb000249    bl  6ca8 <_ZN6Thread3Ip63MplC1Ev>
        6380:   e8bd4008    pop {r3, lr}
        6384:   e12fff1e    bx  lr
        6388:   200068e0    .word   0x200068e0
    

    But same as above, since objdump is trying to interpret 32 bit hex probably these are not the right instructions.

    What's your objdump command?
    If you type disassemble my_function in gdb you will get a proper
    disassembly, at least if you're starting it with make debug in RIOT.
    Den 27 jun 2016 18:35 skrev "jia200x" notifications@github.com:

    I mean, for the same address GDB and objdump show different instructions.

    Thumb instruction set of cortex-m0 looks usually looks like:

    1af4: b538 push {r3, r4, r5, lr}
    1af6: 6001 str r1, [r0, #0]
    1af8: 6901 ldr r1, [r0, #16]
    1afa: 6943 ldr r3, [r0, #20]

    16 bit instructions (sometimes 32, but it depends on the instruction).

    Some of the OpenThread functions are being shown like this in the assembly
    file:

    1f70: e3520000 cmp r2, #0
    1f74: 13510000 cmpne r1, #0
    1f78: e92d4010 push {r4, lr}
    1f7c: e1a0c001 mov ip, r1

    So basically objdump is trying to interpret these 32 bits instructions
    (when they should be 16 bit). It's an objdump problem?

    The pointer that is causing problems is the first one (0x6374), that
    points here:

    6374: e92d4008 push {r3, lr}
    6378: e59f0008 ldr r0, [pc, #8] ; 6388 <_GLOBAL__sub_I__ZN6Thread3Ip63Ip610NewMessageEt+0x14>
    637c: eb000249 bl 6ca8 <_ZN6Thread3Ip63MplC1Ev>
    6380: e8bd4008 pop {r3, lr}
    6384: e12fff1e bx lr
    6388: 200068e0 .word 0x200068e0

    But same as above, since objdump is trying to interpret 32 bit hex
    probably these are not the right instructions.

    You are receiving this because you were mentioned.
    Reply to this email directly, view it on GitHub
    #5563 (comment), or mute
    the thread
    https://github.com/notifications/unsubscribe/AATYQpePo0WUvlubGaJmamZ30_D8JZp0ks5qP_vOgaJpZM4I63gr

    I'm using:
    arm-none-eabi-objdump ./bin/samr21-xpro/openthread_example.elf -d

    the dissasemble function shows:

    0x00006374 <+0>:    ands    r0, r1
       0x00006376 <+2>: stmdb   sp!, {r3}
       0x0000637a <+6>: b.n 0x5ebc <Thread::Ip6::HandlePayload(Thread::Message&, Thread::Ip6::MessageInfo&, unsigned char)+8>
       0x0000637c <+8>: lsls    r1, r1, #9
       0x0000637e <+10>:    add.w   r0, r0, r8, lsl #16
       0x00006382 <+14>:    ldmia.w sp!, {r1, r2, r3, r4, r8, r9, r10, r11, r12, sp, lr, pc}
       0x00006386 <+18>:    b.n 0x65e8 <Thread::Ip6::Address::IsLinkLocalAllRoutersMulticast() const+48>
       0x00006388 <+20>:    ldr r0, [r4, #12]
       0x0000638a <+22>:    movs    r0, #0
    

    So, it seems is crashing in the stmdb line.

    PS: I find a little bit weird the b.n 0x5ebc. Is pointing to the middle of a function that shouldn't be called during static init. Is that OK?

    00005eb4 <_ZN6Thread3Ip613HandlePayloadERNS_7MessageERNS0_11MessageInfoEh>:
    5eb4: e3520011 cmp r2, #17
    5eb8: e92d4008 push {r3, lr}
    5ebc: 0a000007 beq 5ee0 <_ZN6Thread3Ip613HandlePayloadERNS_7MessageERNS0_11MessageInfoEh+0x2c>
    5ec0: e352003a cmp r2, #58 ; 0x3a
    5ec4: 0a000002 beq 5ed4 <_ZN6Thread3Ip613HandlePayloadERNS_7MessageERNS0_11MessageInfoEh+0x20>
    5ec8: e3a00000 mov r0, #0
    5ecc: e8bd4008 pop {r3, lr}
    5ed0: e12fff1e bx lr
    5ed4: eb00375a bl 13c44 <_ZN6Thread3Ip64Icmp13HandleMessageERNS_7MessageERNS0_11MessageInfoE>
    5ed8: e8bd4008 pop {r3, lr}
    5edc: e12fff1e bx lr
    5ee0: eb0006e5 bl 7a7c <_ZN6Thread3Ip63Udp13HandleMessageERNS_7MessageERNS0_11MessageInfoE>
    5ee4: e8bd4008 pop {r3, lr}
    5ee8: e12fff1e bx lr

    What's the value of the stack pointer register when the stmdb instruction
    is executed?
    Btw, could you post the latest output from the hard fault handler? The
    register dump including the extra status registers (bfar etc)
    Den 27 jun 2016 19:09 skrev "jia200x" notifications@github.com:

    I'm using:
    arm-none-eabi-objdump ./bin/samr21-xpro/openthread_example.elf -d

    the dissasemble function shows:

    0x00006374 <+0>: ands r0, r1
    0x00006376 <+2>: stmdb sp!, {r3}
    0x0000637a <+6>: b.n 0x5ebc
    <Thread::Ip6::HandlePayload(Thread::Message&,
    Thread::Ip6::MessageInfo&, unsigned char)+8>
    0x0000637c <+8>: lsls r1, r1, #9
    0x0000637e <+10>: add.w r0, r0, r8, lsl #16
    0x00006382 <+14>: ldmia.w sp!, {r1, r2, r3, r4, r8, r9, r10,
    r11, r12, sp, lr, pc}
    0x00006386 <+18>: b.n 0x65e8
    <Thread::Ip6::Address::IsLinkLocalAllRoutersMulticast() const+48>
    0x00006388 <+20>: ldr r0, [r4, #12]
    0x0000638a <+22>: movs r0, #0

    So, it seems is crashing in the stmdb line.

    You are receiving this because you were mentioned.

    Reply to this email directly, view it on GitHub
    #5563 (comment), or mute
    the thread
    https://github.com/notifications/unsubscribe/AATYQrmXrQ8-Tt_NJTOsEJmhv_red9z0ks5qQAPBgaJpZM4I63gr
    This is the crash:

    2016-06-28 11:50:36,228 - INFO # Context before hardfault:
    2016-06-28 11:50:36,229 - INFO #    r0: 0x00000000
    2016-06-28 11:50:36,231 - INFO #    r1: 0x00000655
    2016-06-28 11:50:36,232 - INFO #    r2: 0x00000000
    2016-06-28 11:50:36,234 - INFO #    r3: 0x00006374
    2016-06-28 11:50:36,235 - INFO #   r12: 0x7dbfff7f
    2016-06-28 11:50:36,237 - INFO #    lr: 0x00025d17
    2016-06-28 11:50:36,238 - INFO #    pc: 0x00006374
    2016-06-28 11:50:36,240 - INFO #   psr: 0x40000000
    2016-06-28 11:50:36,240 - INFO # 
    2016-06-28 11:50:36,241 - INFO # Misc
    2016-06-28 11:50:36,242 - INFO # EXC_RET: 0xfffffff9
    2016-06-28 11:50:36,246 - INFO # Attempting to reconstruct state for debugging...
    2016-06-28 11:50:36,247 - INFO # In GDB:
    2016-06-28 11:50:36,249 - INFO #   set $pc=0x6374
    2016-06-28 11:50:36,250 - INFO #   frame 0
    2016-06-28 11:50:36,250 - INFO #   bt
    

    Before executing the stmdb instructions, registers ar:

    r0             0x0      0
    r1             0x655    1621
    r2             0x0      0
    r3             0x6374   25460
    r4             0x0      0
    r5             0x2      2
    r6             0x2c55c  181596
    r7             0xfef19bdf       -17720353
    r8             0xef7cf7ff       -277022721
    r9             0xbffebfd0       -1073823792
    r10            0xd7bebfbe       -675364930
    r11            0xfbfe3f5e       -67223714
    r12            0x7dbfff7f       2109734783
    sp             0x200077b0       0x200077b0 <isr_stack+488>
    lr             0x25d17  154903
    pc             0x6374   0x6374 <_GLOBAL__sub_I__ZN6Thread3Ip63Ip610NewMessageEt()>
    xPSR           0x40000000       1073741824
    

    About the stack pointer:
    0x200077b0 <isr_stack+488>: 0x00000000

    It's 0. Where's the size of stack defined?

    The SAMR21 doesn't have extended fault registers.

    Cheers

    I compiled OpenThread without RIOT and checked some similar functions.
    For example, these are the instructions of HandleAckTimer in OpenThread example:

    0020148c <_ZN6Thread3Mac3Mac14HandleAckTimerEv>:
      20148c:       b510            push    {r4, lr}
      20148e:       4604            mov     r4, r0
      201490:       f00c fc3e       bl      20dd10 <otPlatRadioIdle>
      201494:       f894 3090       ldrb.w  r3, [r4, #144]  ; 0x90
      201498:       2b01            cmp     r3, #1
      20149a:       d002            beq.n   2014a2 <_ZN6Thread3Mac3Mac14HandleAckTimerEv+0x16>
      20149c:       2b03            cmp     r3, #3
      20149e:       d022            beq.n   2014e6 <_ZN6Thread3Mac3Mac14HandleAckTimerEv+0x5a>
      2014a0:       e02b            b.n     2014fa <_ZN6Thread3Mac3Mac14HandleAckTimerEv+0x6e>
    

    When compiling with RIOT, these are the instructions.

    0000456c <_ZN6Thread3Mac3Mac14HandleAckTimerEv>: 456c: e92d4010 push {r4, lr} 4570: e1a04000 mov r4, r0 4574: eb008b79 bl 27360 <__otPlatRadioIdle_from_arm> 4578: e5d43090 ldrb r3, [r4, #144] ; 0x90 457c: e3530001 cmp r3, #1 4580: 0a00000b beq 45b4 <_ZN6Thread3Mac3Mac14HandleAckTimerEv+0x48> 4584: e3530003 cmp r3, #3 4588: 0a000000 beq 4590 <_ZN6Thread3Mac3Mac14HandleAckTimerEv+0x24> 458c: eafffffe b 458c <_ZN6Thread3Mac3Mac14HandleAckTimerEv+0x20>

    So the instructions are the same, but check the size of the instructions. (E.g b510 and e92d4010 represent push {r4, lr} ) Without RIOT they are 16bit, and with RIOT they are 32bit.

    The fact is that all OpenThread library functions have this 32 bits format. All RIOT functions and hooks have the common 16-bit Thumb format. For example, otPlatSerialSend is a RIOT functions that is supposed to be called by OpenThread.

    000019d4 <otPlatSerialSend>:
        19d4:       b508            push    {r3, lr}
        19d6:       1c02            adds    r2, r0, #0
        19d8:       4803            ldr     r0, [pc, #12]   ; (19e8 <otPlatSerialSend+0x14>)
        19da:       f024 fa87       bl      25eec <iprintf>
        19de:       f025 fc2f       bl      27240 <__otPlatSerialSendDone_from_thumb>
        19e2:       2000            movs    r0, #0
        19e4:       bd08            pop     {r3, pc}
        19e6:       46c0            nop                     ; (mov r8, r8)
        19e8:       00027814        .word   0x00027814
    

    But it seems these 32 bits instructions are being read as 16 bit Thumb instructions (as I showed in a previous comment a b.n 0x5ebcthat is jumping to the middle of a function that shouldn't even be called at start up).

    My pkg script is running OpenThread's makefile and then copying the libraries in BINDIR. Might be something related to that?

    You're right, it seems the open thread code is built in arm mode instead of
    thumb! The addresses in the init array are both even numbers, which means
    that the thumb flag is not set. Check your compilation commands for open
    thread and verify that the -march and/or -mcpu options are correctly set
    (should be the same as for when you compile the rest of RIOT).

    I compiled OpenThread without RIOT and checked some similar functions.
    For example, these are the instructions of HandleAckTimer in OpenThread
    example:

    0020148c <_ZN6Thread3Mac3Mac14HandleAckTimerEv>:
    20148c: b510 push {r4, lr}
    20148e: 4604 mov r4, r0
    201490: f00c fc3e bl 20dd10
    201494: f894 3090 ldrb.w r3, [r4, #144] ; 0x90
    201498: 2b01 cmp r3, #1
    20149a: d002 beq.n 2014a2 <_ZN6Thread3Mac3Mac14HandleAckTimerEv+0x16>
    20149c: 2b03 cmp r3, #3
    20149e: d022 beq.n 2014e6 <_ZN6Thread3Mac3Mac14HandleAckTimerEv+0x5a>
    2014a0: e02b b.n 2014fa <_ZN6Thread3Mac3Mac14HandleAckTimerEv+0x6e>

    When compiling with RIOT, these are the instructions.

    0000456c <_ZN6Thread3Mac3Mac14HandleAckTimerEv>:
    456c: e92d4010 push {r4, lr}
    4570: e1a04000 mov r4, r0
    4574: eb008b79 bl 27360 <__otPlatRadioIdle_from_arm>
    4578: e5d43090 ldrb r3, [r4, #144] ; 0x90
    457c: e3530001 cmp r3, #1
    4580: 0a00000b beq 45b4 <_ZN6Thread3Mac3Mac14HandleAckTimerEv+0x48>
    4584: e3530003 cmp r3, #3
    4588: 0a000000 beq 4590 <_ZN6Thread3Mac3Mac14HandleAckTimerEv+0x24>
    458c: eafffffe b 458c <_ZN6Thread3Mac3Mac14HandleAckTimerEv+0x20>

    So the instructions are the same, but check the size of the instructions.
    (E.g b510 and e92d4010 represent push {r4, lr} ) Without RIOT they are
    16bit, and with RIOT they are 32bit.

    The fact is that all OpenThread library functions have this 32 bits format.
    All RIOT functions and hooks have the common 16-bit Thumb format. For
    example, otPlatSerialSend is a RIOT functions that is supposed to be called
    by OpenThread.

    000019d4 :
    19d4: b508 push {r3, lr}
    19d6: 1c02 adds r2, r0, #0
    19d8: 4803 ldr r0, [pc, #12] ; (19e8 <otPlatSerialSend+0x14>)
    19da: f024 fa87 bl 25eec
    19de: f025 fc2f bl 27240 <__otPlatSerialSendDone_from_thumb>
    19e2: 2000 movs r0, #0
    19e4: bd08 pop {r3, pc}
    19e6: 46c0 nop ; (mov r8, r8)
    19e8: 00027814 .word 0x00027814

    But it seems these 32 bits instructions are being read as 16 bit Thumb
    instructions (as I showed in a previous comment a b.n 0x5ebcthat is jumping
    to the middle of a function that shouldn't even be called at start up).

    My pkg script is running OpenThread's makefile and then copying the
    libraries in BINDIR. Might be something related to that?

    You are receiving this because you were mentioned.
    Reply to this email directly, view it on GitHub
    #5563 (comment), or mute
    the thread
    https://github.com/notifications/unsubscribe/AATYQgolS-Q5uIsh5uAcntW98vnIu_HAks5qQQ8JgaJpZM4I63gr

    Thank you guys. Finally this problem was solved.
    I'm compiling OT with RIOT makefiles instead of using their Make system, so now the right Thumb code is being generated.

    Cheers!

    Just wanted to say something similar. Added some flags to configure openthread (./configure --enable-cli --host=arm-none-eabi CXXFLAGS="-mcpu=cortex-m0plus -mthumb" CFLAGS="-mcpu=cortex-m0plus -mthumb" LDFLAGS="-mcpu=cortex-m0plus -mthumb" CPPFLAGS="-mcpu=cortex-m0plus -mthumb") and get to main now. Still run into a error there on input.