Ashbark Talisman not usable with +Atk Talisman

If I slot the Ashbark Talisman on my second weapon set and slot on the first one a talisman with "+# to Level of all Attack skills", the Ashbark talisman is not usable.

It shows the message "You cannot use this item. Its stats will be ignored".

I tried swapping it around (moving Ashbark to the first weapon set), it still shows the error message, but this time on the other talisman.

So it shows always the message on the talisman in the second weapon slot.

Every other talisman without "+# to Level of all Attack skills" works fine.