That "island" is not an island, it is the Corunian spit, which at its southern end runs over the Russian border, being in the Kaliningrad oblast, the separate area of land, formerly part of East Prussia in Germany until taken by the USSR at the end of the Second World War.
Yes - it's a peninsula in the form of a long spit, with the land entry being via Kaliningrad - so the only way to it within Lithuania is by boat.
There is a very pretty village on the eastern side called Nida, that is the former home of the German writer Thomas Mann. Just inside the Lithuanian border there is a low hill with a decorative sundial laid out on the summit from which you can look out over the Russian border.
Klaipeda itself has an interesting history. It was also ceded to the USSR at the end of the war, but ended up in the Lithuanian SSR. It was formerly known as Memel, and was the northern most city in pre-war Germany, and previously a Hanseatic city. You can see all those influences in the city, even if it's not the prettiest Baltic city. As you say, worth a visit, but you'll probably need to hire a car to make the most of the Corunian spit. Good for birding too!
There is a bus service from the passenger ferry (connects off at least alternate ferries all way down to Nida) and I understand there's also a bus from central bus station that takes the vehicle ferry that's about 5kms down but, agreed, a car for a short stop at spots along the way would be useful.
The forest and outer beach, even walking into them from the foot passenger terminal with its hordes of children in school parties, was an oasis of quiet and beauty - tranquil - and, yes, I could have walked for miles. Ahead of me on the path I saw a deer - no discernible photo (the technology of the eye is better at seeing these things than the technology of a phone camera to this day).
I could have spent another 2 days in Klaipeda ... but then I could have spent longer in Amsterdam, in Rostock, and in the other placates along the way. Two full days in Vilnius now and we are booked on one of the two international trains from here on Tuesday.