If your PS4 link is slow make sure your router supports IP Fragmentation. You can check this by running an Internet Connection Test from Vita Settings. If it doesn't tell you there's an issue with an asterisk than your router supports IP Fragmentation.
It is I think perhaps a big jump in the playability of the PS4 Link over a router and network vs the direct connect.
Once your network connection is looking good make sure you disable direct connect as it doesn't roam well between the two.