Computing Q = d·G takes a few hundred point operations with double-and-add or a ladder; reversing it has no known efficient classical algorithm on well-chosen curves. The best generic attack, Pollard's rho, needs about √(πn/4) group operations for a group of prime order n, so a 256-bit curve offers about 128-bit security. Index-calculus methods that make finite-field discrete logs and factoring sub-exponential do not apply to prime-field curves, which is why EC keys are so much shorter than RSA keys.
Curves are chosen to avoid known shortcuts: anomalous curves (Smart's attack), curves with a small embedding degree (MOV pairing transfer, which is exactly what pairing-friendly curves accept on purpose), group orders with only small prime factors (Pohlig–Hellman, hence the prime order n and a small cofactor) and weak twists. The largest publicly solved generic instance on a prime-field curve is 112 bits (secp112r1, solved in 2009 on a cluster of PlayStation 3 consoles).
Shor's algorithm solves the ECDLP in polynomial time on a large fault-tolerant quantum computer, which is why every elliptic-curve scheme is on NIST's post-quantum transition list.