It seems that I may not connect a same sip extension with two sip phones at a time. Is there anything I could do about it, or do I have to use a follow-me solution?
Not possible due to registration behaviours and not necessary.
Maybe the only thing that you need is to create two extensions and a call group. If you want the two extensions to appears as the call group when calling freepbx lets you assign what to send to identify.