As long as you're fairly smooth with your riding and aren't doing anything too stupid it should hold up O.K. Most bikes are overkill for their intended purpose except maybe world cup xc bikes or redbull rampage bikes. Get a bike that is good for the riding you'll be doing 90% of the time not one that is built to do those extreme bits you do every now and then.
If you're going to be doing some downhill mixed with other riding I'd buy a bike with at least 5" travel preferably six. You can pick up older enduros with 150mm travel for not much, they're not light but they're a lot lighter than 40lbs.