Why I Broke Up With My Financial Advisor
This is the story about my partnership, relationship, and subsequent breakup from my financial advisor.
This post was originally part of my review of the Fire Your Financial Advisor course from The White Coat Investor. Click here to read my review. I separated them as I felt they deserved their own post (and it was really long).
Paranoia
I began learning investing from my father at a young age and, I obtained a business degree in undergrad despite going into medicine. I’m naturally inclined to think about personal and business finances. So I had a natural desire to take the lead on it when I became married and had a real job.
While I didn’t have a natural idea on how to communicate ideas to my wife, I did generally know the broad strokes of what we should do.
After the birth of our first child, Rogue One, I became paranoid and wanted someone to make sure we were “on the right track.” Kids. Change. Everything.
So I hired a financial advisor.
It was a well-known company — supposedly with more physician clients than any firm in the country. The founder/owner and his younger brother were going to be my advisers (Sr. Brother and Jr. Brother). I had even heard about them from someone I trust — my older brother, who had briefly worked with Sr. Brother several years prior.
Sign up to receive email notifications of posts. Follow me on X, Facebook, or Instagram for entertainment and updates of new posts. Feel free to email me with comments, questions, or to ask about site sponsorship or public speaking.
So what could go wrong?
What Went Sort Of Right
They told us we were doing great — our loan debt was low and we had already maxed our Roth IRAs every year . Compared to most recent med school grads we were doing phenomenal. So we got a pat on the back. The advisor only had two concrete, helpful pieces of advice:
- Obtain a personal, occupation specific, disability insurance policy
- Complete basic estate planning (will, revocable trust, power of attorney, etc).
Completing those was easy — they helped with the insurance, a lawyer friend gave me a discount on obtaining the estate documents.
Jr. Brother convinced me to use their investment management services. In exchange for paying an assets under management (AUM) fees, they would do all the investing. With AUM fees, no matter how much is invested and regardless of if the value goes up or down, they take a fixed % of your invested assets. If you have a tiny portfolio it feels okay
They also gave me access to Dimensional Fund Advisors (DFA) mutual funds/ETFs. They said DFA funds would give me an edge over the standard Vanguard funds. That topic is hotly debated in investing circles.
They charged me ~1.5% AUM fee. For those not familiar with what’s the magnitude of fees — THAT’S A LOT. Over time an advisory fee that high can eat up large amounts of your portfolio, more than 30% over 30 years, sometimes way more.
I didn’t do the math of how much advisory fees could cost long-term. The White Coat Investor hadn’t launched. Physician On Fire wasn’t yet financially independent and writing about it.
But I quickly saw I was paying a lot with very dubious benefit. The company put almost no effort into the investments. They just set an allocation and every few months checked in and did a little re-balancing. They were doing what I was doing — a solid, low-cost, index fund based approach.
They put as little effort into managing the investments as I did, but now I was paying a high fee.
Whenever I asked them to show me they were outperforming Vanguard, they were unable to do so. I was later told the fee was for the convenience and advice and they would not necessarily help me outperform Vanguard.
The company also provided other services that were theoretically covered by my AUM fee and an initial contract fee. So I asked tons of questions about refinancing home loans, reviewing my first attending physician contract, etc.
Why We Fired Each Other
They never put effort into helping us define savings or retirement goals or planning after the initial intake assessment.
I kept questioning the value of their investing products. I kept asking questions that they said were covered by their services. I also got annoyed when I would ask a question that needed an answer (i.e. within a week) and 2 weeks would go by without acknowledging my question.
My lead advisor, Jr. Brother, said they weren’t able to meet my needs and that I was asking for things not covered by my fees, and even said I had not paid what I was supposed to pay (trust me, I had paid everything we agreed on). He said we had to narrow our working relationship — he would no longer answer questions, but instead would provide a quarterly meeting to review my investments and nothing else. For the same massive fee.
I responded back — quite outraged — and accused them of a contract breach. I couldn’t find my original contract, but I fired them. A short time after we severed our relationship I found our original contract, which I had lost, and sent it to them.
I requested they partially refunded my fees because of services not delivered, and said if they took that step I would consider the dispute settled and would not pursue a formal complaint, which I had considered. The amount of money was tiny to them, a lot to me. They sent me a check. Because I chose not to file a formal complaint, I am not putting their name in this blog post.
Also to be fair — Sr. Brother was gracious at the end of the process. It was Jr. Brother where I encountered difficulty. He has recently left the company, though I do not know why.
Ever since I’ve immersed myself more by reading blogs and the occasional book. I hate reading investing books, so I mostly read websites and articles online, but there are tons of good books out there.
A good deal of what many advisors pay to do is quite easy, even for a novice. Managing investments, picking a home loan, etc. That part is not difficult for most well-educated people. It’s comprehensive planning, setting realistic goals, and avoiding doing stupid stuff where they earn their fee.
I still constantly ask for advice — I just don’t pay someone most of the time. I ask on websites and forums and pester people I know who know more than me, and people I don’t know but who are willing to help (like the White Coat Investor himself by email or website).
Do you have a financial advisor breakup story? Share it below.

Thanks for your story. While reading this I thought about what would happen if I told some of my needier patients that we “had to narrow our working relationship” and that I would no longer answer their questions but would meet occasionally meet and make sure their labs were ok. The thought made me laugh out loud at the absurdity.
I also fired my financial advisor a few years ago, and wish I had done it sooner. My experience was better than yours, but I also felt like I wasn’t getting my money’s worth–like you said, it’s not that hard to learn to do most of this stuff on your own. And he just didn’t seem to get me–e.g. he didn’t understand why I was living on less than half of what I earned and investing the rest (even though that meant more $ for him), and didn’t understand that I don’t care about status at all but care about my future security and ability to (hopefully) someday have some sort of philanthropic foundation a lot.
The conduct of the advisor I felt was fairly egregious — it was far from professional and I am still surprised how he chose to approach things. Our advisor wasn’t really even helping define goals or decide how to work towards them. He provided great value in the 2 areas I mentioned, but ignored on what would’ve been very helpful to me at the time
I have left two Advisors (and various financial products salespeople) over the years, most recently in 2011/4.
In that last case, I was working with a very reputable firm (based in our home town, but with a national presence) that our group contracted to manage and invest our self-directed retirement plans. The firm also recruited our individual business, and my wife and I met with them and signed up.
We started in late 2007, and I had the mindset, or perhaps they planted this seed), that if we invested in their favored DFA products, over time, the excess returns from the small-value tilting would cover their fees. I remember in late 2011, looking at my 5 year returns, and due to the market swoon, the returns were 0.2% annualized, net of fees.
I went back and added up all of the fees for the various accounts and AUM fees (0.35% for retirement plan funds, 0.8% for personal assets), and I had paid over $50k in fees.
That is not to say that they did not provide good advice, service, reviewed and tweaked insurances, started me on a backdoor Roth Plan, etc., but after the first two years, we were meeting for lunch once a year and they were collecting fees. I could no longer stand it. There were a couple minor snafus and annoyances that also influenced my decision.
I moved out my non-retirement plan assets first, in 2011, and took ove the retirement plan assets (after some negotiation with my practice) in 2014.
Somewhere between 2011 and 2014, I was at a dinner party seated next to a financial advisor, and we talked about my experience and time line. He confided to me that it is well known in the industry that it is hard to keep people longer than 5-7 years.
I can imagine they have a high turnover, but I am sure there are some cash cows that just don’t pay attention to the fees lost or the impact it has. As WCI also points out, a lot of doctors are still not wanting to learn the basics
My experience is similar to Vagabonds. I have fired 2 money guys. Both were commissioned stock brokers. I had it when I showed the Merrill Lynch guy the ER costs I was paying from the Personal Capital phone app he told me $29000 was not bad for an account of my size. I moved to Vanguard.
Oof. Your portfolio is larger than mine but I can’t imagine that high of a few
I’m lucky – I’ve never been sucked into getting a financial advisor – a combination of skepticism and unconscious bias actually working in my favour this time!
It did work well for you. I admit I went of my own volition to the advisor, and the company itself is mostly reputable and provided reasonable advice. But they clearly wanted a client that wouldn’t ask too many questions, or one with a much higher net worth to justify their time.
So if you’re not rich enough to justify their precious time, then you should sit down and shut up?
That kind of attitude pisses me off to no end! If you’re good enough to take my money, you better answer my questions and/or provide me a service of some value!
This is EXACTLY why so many people are DIY investing with Vanguard these days…
Froogal — I believe that is the message the advisor was trying to convey to me. He did not specify my net worth as a reason to not answer my questions, however I would be shocked if he said the same thing to someone with my current investment assets, let alone someone with much higher assets, in the same situation and the same contract.
I have never had a financial adviser. I feel like I should, but I am so skeptical of getting stuck with a salesperson instead of a true adviser. The reality is that a large percentage of financial advisers are just going to sell you a product and can’t predict the market any better than we can. Really good fiduciary advisers are hard to find.
A financial advisor can have a lot of value, but I encourage you to learn the basics on your own. If you are starting off and just want to figure out the basics of investing or debt management, you don’t need an advisor. If you need comprehensive planning or a complex portfolio or a personal relationship, then an advisor can be great.