Hey Amelia- long time KP sufferer here. I have it on the top of my arms. I've tried lots of things- BP, Retrieve cream, coconut oil, exfoliating, jojoba oil... recently I've been using super thick emu oil moisturiser, which is readily available here in Australia (not sure if it is in other countries). Anyway, it's got a super high fat content. Common knowledge is that KP is helped by keeping skin moist- dry skin worsens the keratin plugs.
So go for a moisturiser with a really high fat content. Not sure why but coconut oil wasn't as effective, but I have heard that coconut oil actually dries your skin because it pulls out your skin's natural oils... Also, it just never seems to absorb properly into my skin.
So right now I'm moisturising a couple of times a day with the emu oil and some times I manually dislodge the KP plugs... this is a bad habit though!