Ireland, Sweden and Austria are all both neutral and EU member states: but can you really be both, and what is the value of policies of neutrality and alternative foreign policies? (My main focus will be on Ireland, since I'm more familiar with its policy goals).
Neutrality is a very value-driven foreign policy, and there are several reasons why states become neutral; most of these are tied up in history and identity as well as in the national interest. In Ireland a policy of neutrality underlines the independence of the state, running on the 1940s logic that the ability to stay clear of great power wars was a mark of sovereignty. In Austria, neutrality was necessary to ensure the country's freedom and sovereignty in the middle of Cold War Europe.
Both these policies were the best policies for their time: Ireland, having just been through a war of independence, a civil war, and a trade war, was in no position to enter a world war - especially if it meant opening Ireland to the British military when the conflict of the last 20 years had been about getting them out in the first place. Internal stability and external independence depended on neutrality (and on Ireland being lucky in the course of the war). For Austria, preventing the division of the country between east and west in the same manner as Germany depended on constitutional guarantees that the Second Austrian Republic would be a neutral state.
But, for Ireland at least, neutrality wasn't meant to be permanent; it was just wartime policy. Post-War Ireland tried to ally itself with America, but America turned Ireland down: it was either the NATO alliance or no alliance. Since joining NATO would mean recognising British sovereignty over Northern Ireland, the republic decided not to join.
So what does the EU mean for neutrality, and what does the EU mean for the calculation of national interests when it comes to neutrality?
Under the NATO umbrella, the EU has created a zone of peace, so Europe is not split between 2 antagonistic blocs. This doesn't mean that there's no value in being neutral, but it does change the calculation: the policy goals of neutral states can be more effectively achieved through cooperation with fellow European states without the risks that come with associating with an alliance bloc in tense international circumstance. For Ireland, the main foreign policy goals, apart from a general policy of trying to prevent conflict through the UN and other international organisations, is non-proliferation, disarmament of inhumane weapons (e.g. cluster mines), and conflict resolution through international peacekeeping mission. (Leaving aside the goal of eventual reunification).
The EU, and the interconnectedness with other European states, can be an important factor in all of these aims, and in getting them more widely recognised and more effective action taken on a regional and global level.
The internal market also means that member states share, directly, the same interests in several fields. The free movement of people means that immigration is a shared issue, even for non-Schengen states. Connected to this is the stability and prosperity of the countries neighbouring the EU. Immediately there is a need for a common approach to development aid, trading policy (which is common in any case in the EU), as well as the need for a more political foreign policy voice in the region: whenever trouble breaks out, we are affected; US policy in the region therefore also has an affect on us. The best way of ensuring an effective policy is to have a common policy. Ireland has been involved in helping promote stability in the region along with other European states, particularly in the EUFor in Chad. Energy issues and environmental issue are also areas which require a high degree of coordination.
These goals do not require the sort of militarisation of the EU in the way a lot of neutrality supporters in Ireland fear. See some good defences of the foreign policy role of the EU here and the EDA here.
In this context, the question of whether a neutral or non-aligned or even an allied stance would be the best for pursuing both national and common interests should be asked. Some neutral supporters have (successfully) tried to frame the argument by portraying the choice as one between neutrality (of which Ireland's is in any case doubtful - it even has its own wikipedia page and an almost separate definition from classical neutrality) and a militaristic super-state (possibly practicing conscription). Ignoring the fact that many European states are phasing or have phased out conscription, why should the debate in Ireland, and possibly in the other neutral European states this is similar, be limited to these 2 choices?
In my opinion, the existence of the internal market and membership of the EU has changed the political context for neutral European states. This doesn't automatically mean that military integration should be adopted, but a range of options should be discussed. Still, integration to some degree of foreign policy can provide some clear advantages. Of course, it will never be a simple calculation of interest; the politics of identity will mean that any discussion of foreign policy in neutral states will be fraught with emotional arguments.
Note: Fine Gael is the only Irish party that has a policy of cooperation in a European defence policy.