There is no difference to the outcome, plot or missions of the game if you pick the bird or the cage. There is a visual difference later on in that Elizabeth’s broach will be whatever you do not choose. So if you chose bird, she will be wearing a broach with a cage on it.
I won’t say when or why that is for reason of spoilers.
There is no achievement attached to either choice either.