A harder question than you think. No matter what being fluent means to you, it's always something that bothers us when we pick up a new language.
Yes, learning a language can be scary. That's why I like to break it down into smaller, bite size pieces.
By looking at fluency as a process and not as a final destination, language learning becomes easier.
Since I started teaching languages, I have been asked "when do you feel that you are fluent in a language?"
Instead of aiming for fluency, aim to have a strong basis of what you need to be fluent in the section of the language that you want.
This past summer, I was staying at my good friend Rafael's house in Paris when we came across a small plumbing problem. After a painful conversation with the French plumber who knew little to no English, I realized something... I'm not fluent in plumbing French. If I want to remodel an apartment in Paris someday, I'll have learn this side of French all over again.
I am fluent in other sections of French. I am fluent in asking for directions, talking about what I do, talking about my friends and family, and other typical conversations. Sounds a bit weird, huh?
My suggestion: picture yourself being fluent in a particular situation instead of on being fluent in the entire language. Once you conquer one situation, then you can move on to the next one.