With Tuesday’s Supreme Court ruling upholding Trump’s Muslim ban, Wednesday’s decision attacking public sector unions, and Justice Anthony Kennedy’s announcement that he’s retiring, it is time to push a once-marginal idea to the top of the agenda: pack the Supreme Court. The conservative majority’s support of Trumpism and opposition to progressive objectives means it will pose a barrier to the agenda of even the most left-leaning president and Congress. This barrier must be confronted head on.

Today, such ideas smell toxic to the average Democratic elected official — recall Al Gore’s surrender following Bush v. Gore in 2000, despite later evidence that he was the rightful winner of that election. But such deference wasn’t always the norm. There’s nothing in the Constitution specifying the number of judges, the precise organization of the federal judiciary, or even that the Court be empowered to review acts of Congress.

All of this was laid out in the early years of the American republic, through a series of judiciary acts in Congress and incremental power grabs by the Court. Political scientist Stephen Engel counts 833 efforts by Congress between 1800 and 1982 to reorganize the judiciary to constrain or channel political interests. And all of the successful impeachment convictions in US history (eight) were against judges, who also represented a majority of the nineteen impeachments handed down by the House of Representatives. This makes sense: the monarchy-averse founders were so distressed by even four-year presidential terms that they gave Congress the ability to remove them mid-course, so of course they would feel doubly cautious about lifetime judicial appointments — one of only two job specifications (the other being a non-diminishable salary).

From the time the justices unilaterally asserted their power to strike down legislation in 1802, a democracy-eroding judicial supremacy has been an ever-present danger. One of the most significant confrontations came in 1937, when the Franklin D. Roosevelt administration decided to pick a fight with the Court.

The stakes couldn’t have been higher. Since the 1890s, the so-called “Lochner era” Court had been dominated by justices bent on crimping the growth of the administrative state through libertarian interpretations of substantive due process. The 1905 case that gave the era its name had voided New York’s maximum hours law for bakers, on the grounds that it interfered with workers that might wish to labor for longer. This overall orientation carried into the 1930s. According to a count from political scientist Randall Calvert, from 1934 to 1936, the Supreme Court heard fourteen cases against New Deal policies and ruled against nine. The casualties included the 1933 National Industrial Recovery Act (NIRA) — which had promised, among other things, to bring unions to much of American industry. (The execution had not been to labor’s liking, but that’s another story.)

The early New Deal-era Court’s hostility to government activism did not neatly cut across partisan lines: the four-member conservative bloc included two Democrats, and the two swing justices were Republican. Even the three ostensibly liberal justices were unreliable allies. All three had joined three unanimous rulings against the New Deal issued on May 27, 1935, which came to be called Black Monday. These included decisions against NIRA (A.L.A. Schechter Poultry Corporation v. United States) and FDR’s firing of a conservative commissioner on the Federal Trade Commission (Humphrey’s Ex’r v. United States). Liberal lion Louis Brandeis even authored the decision against mortgage moratoria, finding them to be an unconstitutional seizure of banks’ private property (Louisville Joint Stock Land Bank v. Radford). Another liberal, Benjamin Cardozo, was the only one to dissent in an earlier NIRA case (Panama Refining Co. v. Ryan), a dissent he described as but a “narrow” point of difference with the other eight justices.

Had such precedents been allowed to stand, much of the basic federal legislation we took for granted (at least until the Roberts Court) would have been impossible to build. FDR had long sensed this, considering early in his first term a number of options to restrict the Court’s judicial review powers. Yet as historian William Leuchtenburg reports in his book on the era, public opinion was generally sour on these ideas — the Court was associated with the integrity of the Constitution itself. Nonetheless, FDR had triumphed in the 1936 elections, winning all but two states and the highest share of the popular vote in the history of the two-party system. He returned to office determined to find a way to unstick the New Deal.

On February 5, 1937, Roosevelt proposed the Judicial Procedures Reform Act, a bill that would have empowered him to appoint a new justice for every sitting one who refused to retire within six months of his seventieth birthday. At the time, six justices were already older than that, so the move would have expanded the Court to fifteen. The idea of changing the number of justices on the Court was not unprecedented. In 1789, there were initially only six justices, and statutes over the years had specified five, seven, eight, and even ten seats. Nine justices were not the norm until 1869 — when FDR’s older half-brother James was already fifteen years old.

The president had timed that announcement to precede the February 8 oral arguments in N.L.R.B. v. Jones & Laughlin Steel Corp., which dealt specifically with the 1935 Wagner Act’s prohibition on discrimination against union members and more generally whether the federal government could regulate labor relations. Roosevelt’s March 9 radio fireside chat further addressed his aims, noting how his 1933 financial crisis response had only narrowly survived Court review:

The change of one vote would have thrown all the affairs of this great nation back into hopeless chaos. In effect, four justices ruled that the right under a private contract to exact a pound of flesh was more sacred than the main objectives of the Constitution to establish an enduring nation. . . . The Court has been acting not as a judicial body, but as a policymaking body. . . . We have, therefore, reached the point as a nation where we must take action to save the Constitution from the Court and the Court from itself.

The effect of FDR’s threat was immediate. Within weeks, in what came to be called “the switch in time that saved nine,” the two swing votes joined with the three liberals to uphold the Wagner Act and Washington State’s minimum wage law. By 1941, emboldened with serious legal protections for the first time, union density doubled (even before the US entry into World War II, when industrial mobilization led to further gains). And by that time, enough retirements and deaths had taken place to allow Roosevelt to appoint a majority of the court. For the rest of Roosevelt’s terms, the Court blessed 100 percent of the New Deal initiatives identified by Calvert.

That’s not to say court-packing is easy. Historians have documented how FDR badly managed public and congressional opinion, and would have had difficulty actually getting a favorable vote on his bill. His ultimate triumph came from being able to wait out the Court by serving more than two terms — something not available to politicians today, despite having relatively young conservative justices like Neil Gorsuch that will be around for decades to come.

Nonetheless, this shouldn’t dissuade us. Political scientist David Faris makes a compelling case that court-packing — along with statehood for D.C. and Puerto Rico and other reforms — amounts to a prerequisite for lasting progressive change. In his new book, It’s Time to Fight Dirty, Faris proposes to expand the roster of the Court to eleven or thirteen immediately, and then pass a law allowing presidents to appoint a new justice every two years. Meanwhile, the most senior justices would be shuffled into a type of emeritus position with lesser responsibilities. The nine most junior cases would do most of the judging, with more senior justices momentarily pulled into duty in the case of a justice’s death.

If this sounds ludicrous, consider the following. First, Senator Mitch McConnell refused to even allow a vote on Merrick Garland — completely shattering whatever bipartisan norms were left around the judiciary. Second, and even more incredibly, prominent Republicans threatened to not confirm any of Hillary Clinton’s nominees had she won — not just in an election year, but ever. Blocking presidents from filling judicial vacancies amounts to neutering the Constitution — unlike the perfectly legal and precedented act of expanding the number of justices. Third, Faris’s four new justices is less than the six that FDR wanted installed, and his greater threat proved just enough to jolt the Court into its rightful role in letting history move forward. And this is the important point: with union density near an all-time low and climate catastrophe on the horizon, future lawmakers will need tools even more robust than what FDR was able to get through — think a Green NIRA on steroids. A handful of justices pulled from Federalist Society debating clubs can’t and shouldn’t get in the way of a more democratic and sustainable economy.

A thoughtful court-packing proposal would ensure that the Court more carefully reflects the mores of the time, rather than shackling democracy to the weight of the past. With inequality and human rights abuses spiraling upward and justices making it all worse, the time to begin mainstreaming an enlarged Court is now.