A taxable brokerage account is one of the most flexible ways to invest your money. And, if you play the game right, you can also reduce taxes in this account in such a way that it may become your favorite type of investing account.
In this episode, we discuss short and long term capital gains, how to minimize capital gains taxes, how to potentially pay zero dollars in capital gains taxes in early retirement, and the importance of understanding how these accounts work.
Transcript
Jimmy Turner MD
So Justin, I’m going to read the question before we dive in and then we can chat about it.
Jimmy Turner MD
Download a free copy!
Â
I think there’s a lot of different directions this could go. So here’s the, here’s the email I got. I thought a good podcast would be to go into a little more depth on taking money out of a taxable account.
Jimmy Turner MD
I know you’ve talked more about using the taxable account due to yields and ability to pull, move that money around for lots of things that may come up. I have some ideas, but what happens when you need that money?
Jimmy Turner MD
What shares should you sell first? What is the cutoff for taxable gains? When and how do you pay that tax? But when it comes to, I’ve got some ideas, but what happens when you need that money? What shares should you sell first?
Jimmy Turner MD
What is the cutoff? So on. And I thought this was a great question for, for lots of different reasons. And listeners probably know at this point that I’m a fan of taxable workers accounts. I know that you’re a fan of taxable workers accounts.
Jimmy Turner MD
And for me, a big reason for that, just new listeners or people that are joining the community, I love the flexibility of a taxable brokerage account. And I’m going to be honest. I actually had a meeting with, with my accountant an hour before this call and they asked me about back to where Roth IRA is, Hey, Jimmy, are you, you know, you still doing that?
Â
Jimmy Turner MD
I said, actually, I’m not, I’m preferentially pacing money into a taxable brokerage account, just for flexibility purposes. And I’ve become such a fan of this that even mathematically that decision probably doesn’t make sense because capital gains tax rates that we’re going to talk about here in a second are 15% on a brokerage account for my family’s income and the early withdrawal penalty for a Roth IRA is 10%.
Jimmy Turner MD
So theoretically I’m costing myself by choosing the brokerage account if I did need the money. But I just love the flexibility of it so much and the ease of access and the ability to just do all of the things that it does that I love, I love the brokerage account and, and there are many other reasons as well, but Justin.
Jimmy Turner MD
You’re a fan, too right?
Justin Harvey CFP
Very much so. I would say the taxable brokerage account is one of these places where a good advisor is going to, you know, you’re talking to somebody who’s as biased as they’re going to get, but this is where they can really earn multiples of their fee through expressing sort of their tax aware investment process through the investments that you hold, through the way that you manage tax loss harvesting,
Justin Harvey CFP
charitable contributions through appreciated security gifting, often in junction with the donor advised fund. And then, you know, sometimes using a taxable brokerage account to fund a backdoor Roth IRA contribution and tax loss harvesting in the process, that’s something else I’ve done and a number of other things.
Justin Harvey CFP
So this actually is also going to relate to what we’re going to talk about next week, which is the cash balance plan for a private practitioner. And the taxable brokerage account is sort of the next thing, the alternative to a cash balance plan.
Justin Harvey CFP
So if you’re asking yourself, should I do a cash balance plan in my practice? If the alternative is a taxable brokerage account, you should really be comparing those two and the merits of tax efficient taxable account investing versus the tax benefit that may or may not come with a cash balance plan in order to understand the big picture.
Justin Harvey CFP
So bottom line, taxable accounts are awesome. If you’re working with a good advisor, they can add a lot of value there. And I’m really excited for this topic today.
Jimmy Turner MD
Yeah, so a good place to start, I think, is capital gains taxes, like what are they? And that sort of thing, because I’m not sure that everybody listening to this is familiar with what they are. They put money into a brokerage account, maybe they have a tax bill at the end of the year, they don’t really understand how that operates.
Jimmy Turner MD
And so big picture, because it’s really easy to get into the weeds on a topic like this. So that’s going to happen inevitably on the show, but we’re going to try to avoid the weeds as much as we can, keep it high level.
Jimmy Turner MD
So capital gains, basically, it’s the taxes that you pay on the money that grows in your brokerage account. And so how that’s determined, short term capital gains, long term capital gains are the two big buckets, it basically comes down to how long you’ve held the asset.
Jimmy Turner MD
So if it’s less than a year, it’s going to be short term capital gains. If it’s more than, I guess, the cutoff is technically 365 days or more, it is long term capital gains. And those are advantageous tax rates.
Jimmy Turner MD
So short term capital gains, it’s like you’re making extra money, took home an extra $100, your marginal tax rates, 30%, you’re going to pay 30% of that $100 that you just made. Not great, right? But if you keep it for 366 days, all of a sudden, it’s 15%.
Jimmy Turner MD
If you’re on the line share of people listening to this show, and 15% on $100, that’s not too bad, right? That’s a pretty reasonable tax rate, particularly when you’re used to paying 25, 30, 40% on taxes for W2 income or other salary that you have coming in.
Jimmy Turner MD
And so big picture, short term, long term, you want to try to avoid short term capital gains when, if at all possible, just because long term capital gains tend to be substantially better. Is that a good way to put it, Justin?
Justin Harvey CFP
Yes, and the important distinction here is to understand the difference between capital gains and earned income, which is like your wages from your job that you get on a W -2. There’s taxes withheld every two weeks, every month or whatever, and that the wages are taxed at what we call ordinary income rates, which are currently 37% as the top margin.
Justin Harvey CFP
So if you bought a bunch of Tesla stock and you held it for a week and it went way up and then you sold it, because it’s a short -term capital gain, if you’re a high income earner, a short -term capital gain is taxed at the ordinary income rate of 37% of the Fed for you.
Justin Harvey CFP
Whereas if you held that for a long term, one year or more, then you would be at a different tax rate that it does get, there’s more complexity to it, but it’s either 15% or 20%, and then there’s an additional charge depending on how much money you make.
Justin Harvey CFP
So it could be more like 25% all in, but it’s still better than 37%. And again, in talking about where does an advisor help with these things, creating turnover, locking in gains on a short -term basis for a high income earner is something you don’t want to do.
Justin Harvey CFP
So this is an important part of monitoring tax -efficient investing in a taxable account.
Jimmy Turner MD
Yeah. So if you’re listening to this and you’re wondering about the cutoffs, because one of the parts of this question was, you know, hey, what are the cutoffs for taxable gains? The three big buckets are 0%, 15%, and 20%.
Jimmy Turner MD
And so I’m just going to mention these numbers really quick, because you’re listening to the show, you likely know what your income is. And so I feel like this is the bit of the weeds that you can listen to this while driving a car, doing the dishes, whatever you happen to be doing, and it’s still going to make sense to you.
Jimmy Turner MD
So for singles, 0% up until an income of $47 ,000, 15% after that, up until $518 ,000. And then 20% is anything above 518. Basically, these are rough numbers, I’m not going to give you down to the dollar amount, because it’s just obnoxious.
Jimmy Turner MD
If you’re married, 0% goes up to $94 ,000. And then from 94 to 583, it is 15%. And then it’s 20% above that. And so if you’re listening to the show, there’s a really good chance that you’re in that 15% bracket.
Jimmy Turner MD
Okay, and this this income is calculated after, you know, you take out what you’re contributing to your 401k, so your gross income minus your 401k or 457, your cash balance plan, all the different tax advantage accounts that you have, potentially your deductions.
Jimmy Turner MD
And so it’s pretty reasonable to be able to get down to that 520 to 580 number for a lot of doctors, and put you in that 15% long term capital gains bracket. So the big cutoffs there are 90 ish if married and then 580, if married for the 0% and the 20% bracket.
Jimmy Turner MD
And I actually remember reading a post, my buddy leaf Daliin listeners may or may not know leaf from position on fire. He’s he sent sold position on fire and moved on to greener pastures. Great guy, he wrote this tax post that I read several years ago, I think the name of it was the tax man leave it.
Jimmy Turner MD
And the purpose of the post is basically to highlight how you could avoid paying any taxes at all on money coming out of your taxable brokerage account in retirement. And he gave lots of different examples for reducing your income that’s used to calculate that to determine your tax, taking your off money out because it’s already been taxed, it’s not going to get taxed again, and just several different maneuvers that could be made to essentially bring home $100 ,000,
Jimmy Turner MD
$120 ,000, $150 ,000 in income, and still pays your percent, because you’re getting your number below that $100 ,000 ish for a married couple. And I remember reading that post and being like, that’s really cool.
Jimmy Turner MD
And someday when I’m in retirement, I want to play games like this so that I don’t have to pay taxes on the money I’m taking for my brokerage account too. So this can be played well if you do
Justin Harvey CFP
Right. Absolutely. And one of the things, the concepts you’re going to want to understand as you’re thinking about tax efficient drawdown and the implications thereof is what’s called a tax lot. So every time you buy a group of securities with a chunk of cash, you create a tax lot.
Justin Harvey CFP
That means a certain security on a certain date at a certain price, you’re setting your basis. So you might have, you might only, if we take the simplest possible example, maybe you’re just holding the S &P ETF and you just buy that IVV every month, you’re dropping in, I don’t know, $10 ,000, buy, buy, buy, buy, $10 ,000 at whatever the price is.
Justin Harvey CFP
If you do that for 20 or 30 years, you’re probably gonna have a lot of money after several decades, but you’re gonna have many, many, many, many different tax lots. So even though you’ve invested a total of $2 million and it’s grown to $5 million, what you’re gonna find is that there’s a bunch of different entry points that the investor has created because of investing over time.
Justin Harvey CFP
So one of the questions that the listener asked is sort of, how do I, you know, minimize taxes on the drawdown? You may have a portfolio that ultimately has grown substantially from $2 to $5 million to $2 million of net investment additions that has grown to $5 of total market value at the time that you want to begin to access it.
Justin Harvey CFP
And yet you will likely have, well, this is less likely if it’s an ETF, it’s more likely if it’s a separately managed account where you’ve got a bunch of individual securities, but you’re probably gonna have some tax lots that actually they may, you might have even purchased that higher than the current market value.
Justin Harvey CFP
What that means is some amount of that $5 million can be accessed not only without adverse tax impact, you may actually be able to tax loss harvest if you bought at a higher price point than the market is today.
Justin Harvey CFP
So this may be a meaningful sum of money that you can access without even worrying about any capital gains. So the first question you always want to ask is if I need an amount of money, do I even need to get into the gains calculation at all?
Justin Harvey CFP
Or can I just take something that’s at a loss because of where that price point was set whenever I created that tax lot and then, you know, create a capital loss.
Jimmy Turner MD
Yeah. So if you’re not in an SMA, which we talked about recently on the direct indexing show for those that don’t know what a separately managed account is the route that I went recently to do this for donor advised fund was used spec ID or specific identification, which is looking at the specific identification of the tax lots.
Jimmy Turner MD
And that allows you to identify the ones that are at a gain or at a loss if you’re trying to tax off harvest or if you’re trying to give to a donor advised fund appreciated gains. The point is, is that it’s all about the basis and where you bought that, that money compared to where the price is now.
Jimmy Turner MD
And so just to give some, some context to this, there is some, some other things too. There are, there is that that’s a, I say this all the time. I’ll be like, Hey, where’s my shoes? And my wife cracks me all the time.
Jimmy Turner MD
She’s like, where are your shoes? So she’s a reading teacher in fourth grade. And for whatever reason I grew up and it was just wares wares, whatever, even if it was plural. So I feel like I just did that publicly to thousands of people, which is, which is great.
Jimmy Turner MD
So yeah, so if you’re sorting through this, right? We talked about the brackets, we talked, we talked about the taxes, long -term, short -term capital gains, when it’s which to sell first, it really kind of does come out into that tax lot, which was a part of this question.
Jimmy Turner MD
And one part of the question, Justin, I’d be curious to know, I didn’t really, I probably needed more context on this and maybe I should have emailed back and gotten that, but the question was when and how do you pay the tax bill when it comes to.
Jimmy Turner MD
And, and so part of me was like, well, you pay your taxes when you pay your tax. I mean, I guess you could pay an estimated taxes on your gains, but I’m personally a big fan of hanging on to as much money as I can and letting it work for me in a, even a high yield account or a money market account, or God forbid I invest it in equities or something like that, because I have enough money laying around to pay the tax bill at the end of the day from other sources.
Jimmy Turner MD
But I thought this was an interesting question, because I think there’s multiple ways to do this.
Justin Harvey CFP
Yeah, so I’ll give you my opinion on this. And this is not tax advice because I’m not a CPA or tax advisor, but a lot of what we do interacts with our clients’ taxes. So the first question I would ask if someone was going to realize a meaningful amount of gains from their taxable investment account is, are we in safe harbor territory?
Justin Harvey CFP
So there’s a couple of different safe harbor tests for income taxes, for federal income taxes. And for our purposes, I’m going to oversimplify and say it’s either 90% of the current year or 110% of the prior year, total liability gets you to safe harbor.
Justin Harvey CFP
So Jimmy, I think we can throw a link in the show notes to the recent conversation we had on this topic, which is really important, especially if you’re a business owner and you’ve got a moving target and you’re doing quarterly payments and you’re trying to sort of land the plane on the aircraft carrier with a crosswind and the seas are sort of making everything shift, it is a moving target.
Justin Harvey CFP
But if you’re going to achieve safe harbor through either one of those tests, then you just pay in April. If you’re going to realize an amount of gains that is so significant that it pushes you out of safe harbor, meaning I was, I expect to make half a million dollars this year and I paid $100 ,000 of taxes last year to the fed.
Justin Harvey CFP
If this year my total paid in is 110 grand, that’s 110% of the prior year is $110 ,000. If I’ve paid that in this year, then I can realize a million dollars of gains and not have any interest or penalties assessed because of the prior year safe harbor that I have met.
Justin Harvey CFP
But if I’m only at like a hundred thousand, then I need to see, am I going to hit the 90% test, which I wouldn’t because I’ve got an extra million dollars of income that I’ve realized and not paid taxes on and I don’t hit the prior year.
Justin Harvey CFP
So you need to make sure that you hit one of those two tests in order to pay in April. If you don’t meet one of those two tests, then you need to do an estimated payment to recognize that, you know, I just made a million bucks of gains that have been unwithheld upon.
Justin Harvey CFP
It’s not like when you get your paycheck and payroll sends money to the fed and the state and then you get the net. It’s if I bought Tesla at a hundred and it grew to 10 ,000 and I sold it, I get $10 ,000 in my brokerage account.
Justin Harvey CFP
I’ve got all of the money and uncle Sam has an IOU on that money, but he doesn’t take the money as soon as I placed the transaction. It’s up to me to understand the safe Harbor and then to make an estimated payment if necessary.
Jimmy Turner MD
Yeah, and that matters because in your example, let’s say we paid 110% of the prior year’s taxes, we had $100 ,000 in 2023. And in 2024, we paid $110 ,000. And all of a sudden, all of that money that’s above that is is free to play with, right?
Jimmy Turner MD
And so that million dollars, you know, you’re going to get taxed on it. So you decide to stuff that million dollars in a high yield savings counter money market account or some other location and you earn money on it for the next 12 months before you pay your taxes, right?
Jimmy Turner MD
Now you have, let’s say a 20% capital gains, you have to pay on that $200 ,000 on that million dollars you earn. But during that time, you let it grow at 5%. And now you have, you know, you’re not paying $200 ,000 anymore, because you’ve now earned $50 ,000 off of that money.
Jimmy Turner MD
Now your tax bill is 150 grand based on the net there. So safe harbor, I’ll link to what that is, maybe in a podcast or not a podcast a post or IRS link to let you read about safe harbor rules, if you want to learn more about that.
Jimmy Turner MD
This comes into play for capital gains, it also comes into play for people that have 1099 or private practice income. And I’ve shared the story on the show before, but it’s just so mind blowing that I share it more than once, which is the the doc that I had a friend who had a partner when he was in private practice that didn’t realize that he had to pay estimated taxes during the year.
Jimmy Turner MD
And so at the end of the year, got there and did have a six figure sum of taxes that were due that he hadn’t paid any money on and he hadn’t saved up for. And so he had to basically create a payment plan with the IRS to pay the money back, obviously with penalties.
Jimmy Turner MD
And in that situation, he just didn’t know about estimated taxes at all. So you need to know about estimated taxes, you need to know about safe harbor rules and how those kind of play into your situation if you’re selling gains in a taxable brokerage account, I think that’s a an important topic.
Justin Harvey CFP
One additional dynamic to be aware of here is, especially if you have a globally diversified allocation in a taxable account, often asset classes move in lockstep, meaning all the international developer, all the emerging markets, or all the, you know, pick your asset class will act similarly over a period of time.
Justin Harvey CFP
So what you may find is if you’re like, Oh yeah, let’s sell the stuff that’s at a loss or at a modest gain in order to keep the tax bill down, that’s all well and good. And frankly, that’s usually the first step.
Justin Harvey CFP
But the next step is what does my portfolio look like now? And often what you’re doing is you’re blowing out of an entire asset class or, or of a only one asset class. Maybe you’re not eliminating the whole asset class, but you’re taking a big bite out of it and your portfolio is now sort of composed of something different than it was before you placed the sales.
Justin Harvey CFP
So you want to think about rebalancing or you want to think about if I’m adding new capital to the portfolio over time to intentionally be managing back to those strategic allocation targets to make sure that you’re not taking some weird kind of risk or really probably what you’re doing is because of the idea of mean reversion, which I call the law of financial gravity.
Justin Harvey CFP
What goes up must come down and also what goes down must return to its long -term average over time, except it doesn’t actually have to. And we don’t know how long of a time that is, but the reason that it interacts with this question at hand is if something is going down, down, down, down, down, down, and then I sell it because I want to tax loss harvest, the mean reversion reality means that if this thing is really depressed in terms of price or this asset class is really depressed in terms of price,
Justin Harvey CFP
I actually want to make sure I’m still exposed to it because that’s where my portfolio is going to get the pop. So if you’re doing this tax loss harvesting in order to capture some cash from your taxable portfolio, just be aware that you may actually be removing yourself from the asset class with sort of the most appealing characteristic just because it’s been hammered all the way into the dirt and you want to make sure that you’re on board that rocket ship whenever it returns to its long -term average.
Jimmy Turner MD
Yeah. Buying stocks on sales is something that I’m a big fan of. And I think the way that you should frame that mentally so that you have the ability to buy when it’s low, it’s also worth noting just because you mentioned it, the tax loss harvesting is something in this situation that would really come into play and help you out a bunch.
Jimmy Turner MD
And so if you had that million dollars, and I’m not saying you’re going to tax loss harvest a million dollars, but the example we gave earlier, you have a million dollars in capital gains, tax loss, harvest a million dollars.
Jimmy Turner MD
You can put those two things together and pay zero taxes in the year in which you’ve done that because you carried those taxes forward. And so tax loss harvesting is another thing to think about specifically, like you just mentioned, Justin.
Jimmy Turner MD
So the timing, you got to consider the tax loss harvesting. You got to consider rebalancing, several things to kind of keep into mind. And then there are ways to help reduce this by playing the game, if you will, particularly in retirement to get yourself back down into a lower bracket.
Jimmy Turner MD
If you are able to, Justin, anything else that we left out? We need to hammer out before we get out, get out of here.
Justin Harvey CFP
There’s a less common circumstance for probably for many of your listeners, but it is a little more common for transitioning physicians who are getting into those like latter fifties early sixties, maybe transitioning out of practice, maybe taking some time to either be retired or semi retired before the social security and RMDs kick in.
Justin Harvey CFP
And you may actually find that as a physician or former physician, you’re in that zone where you’re not going to hit the hundred thousand dollars of income. And so instead of tax loss harvesting, sometimes we see an opportunity for tax gain harvesting where embedded taxable gains can be realized with no tax impact.
Justin Harvey CFP
It’s zero percent taxable gain. And so during that period of time between age 60 and 70, while we’re thinking about Roth conversions and doing the retirement planning and all sort of understanding where the income is going to come from in your latter years, there are times at which it makes sense to realize gains on positions that have appreciated, especially if you’re not going to gift them to reset basis at a higher level so that you can then sell them when you’re 70,
Justin Harvey CFP
80, 90 and realize even less gains at that time. So it’s a a tactical thing. It only applies to some people, but when it does apply, it’s free money. And anytime there’s free money in play, we want to make sure that we bend over and pick it up.
Jimmy Turner MD
Yeah. And you just actually remind me something I wanted to mention on the show before we head out, which is that unlike ordinary income taxes, which are progressive in nature, where you pay at the 0% and then you get to 15 and then 22, like pay pieces along the way, but all of your monies, it’s not like if you land in the 37% tax bracket, you’re paying 37% on all of the money that you’ve earned.
Jimmy Turner MD
Whereas in capital gains territory, it actually does work like that. If you get to the 15% number in terms of income, you’re paying 15% on all of your gains is not like you pay 0% on the gains up until $98 ,000 and then you pay 15.
Jimmy Turner MD
It’s, it’s, it’s, it is a threshold. That’s right.
Justin Harvey CFP
Yeah, an important thing to keep in mind.
Jimmy Turner MD
Thanks to this listener for sending in this question, Jimmy at moneymeetsmedicine .com. If you want to send in your topic, your question, we love answering these because it’s exactly what you want to hear about, which is what we’re all about.
Jimmy Turner MD
Before you head out, don’t forget to go on to moneymeetsmedicine .com, download your free copy of The Physician Philosopher’s Guide to Personal Finance, or look into Own Occupation Disability Insurance, which we can provide from all the companies that do that.
Jimmy Turner MD
That’s really important to us and we’re super passionate about what we’ve been able to accomplish at Money Meets Medicine Disability Insurance, helping docs get the help that they need, and honestly, saving a lot of people from some pretty bad situations along the way.
Jimmy Turner MD
So thank you for those that have trusted us and appreciate you sharing the show and being a part of the community. We will see you next week, cheers. Justin Harvey is a certified financial planner at APM Wealth, where he helps anesthesiologists in pain medicine positions.
Jimmy Turner MD
Dr. Jimmy Turner is a practicing academic anesthesiologist at Wake Forest in North Carolina. He’s also a licensed insurance agent. However, either Justin or Jimmy are your financial planner, investment advisor, or insurance agent.
Jimmy Turner MD
This show is expressly for general education and entertainment purposes only. Nothing should be considered financial advice. All views expressed are solely the views of the guests on the show and do not represent the views or opinions of their employer.
