What you are experiencing is an abutting of your nose threads on the oral mucosa, which can happen as a complication of nose-tip threads.
It is true that the threads will be dissolved by your skin in time, but it can cause a great deal of discomfort in the few months before it does. There is also the possiblity of thread extrusion if there is alot of pressure of the thread on the mucosa.
If the threads are causing you alot of discomfort, I suggest removing the thread via a small puncture hole through the oral mucosa. This will have to be done under aseptic technique and with adequate local anaesthesia.
Hope this helps!