As advocates for medical marijuana gather in Washington, D.C., on Friday for an annual conference, supporters of marijuana legalization are worried.

That's because new U.S. Attorney General Jeff Sessions has been making tough comments about the drug, and there's a lot of uncertainty about how the Trump administration will enforce federal law.

Over his 20 years in the U.S. Senate, Jeff Sessions made no secret of his disdain for marijuana. In his new job as the nation's top federal law enforcement officer, his position on marijuana has not moderated.

As Sessions has said, "I'm definitely not a fan of expanded use of marijuana."

The country, however, is moving in a different direction.

Marijuana is legal in eight states and 28 states have some form of medical marijuana, but the drug remains illegal under federal law.

And that's where the Justice Department, and the new attorney general, have a lot of clout.

"In the world of federal drug trafficking enforcement, marijuana has never been at the top of the list," says John Walsh, who served as U.S. Attorney in Colorado during the Obama years.

The Obama Justice Department responded to state efforts to legalize marijuana by explaining exactly when federal prosecutors would take action.

Selling a joint wasn't enough — instead, prosecutors targeted organized crime rings that shipped the drug to states where it's not legal. Or criminals who marketed to children.

"Heroin or methamphetamine, cocaine, those things have been where the real emphasis and the resources have gone," Walsh says.

He adds that even though Colorado voters legalized marijuana, he actually increased enforcement by making bigger, more important cases.

Walsh says the Trump administration will have to take public opinion into account — just like he did.

"So when we in the U.S. Attorney's office, working with the DEA, looked at a marijuana case here in Colorado, we had to ask ourselves, 'All right, we have to assume if we take this case to trial, seven out of 12 people sitting in the jury box are statistically likely to have voted for marijuana legalization.' "

On Wednesday, Jeff Sessions directed Justice Department lawyers to evaluate marijuana enforcement policy and send him recommendations.

And some state officials are worried. This week the governors of Alaska, Colorado, Oregon and Washington wrote the attorney general.

They asked Sessions and the new Treasury secretary to consult with them before making any changes to regulations or enforcement.

At the White House, press secretary Sean Spicer said recently that the president is sympathetic to people who use marijuana for medical reasons.

He pointed out that Congress has acted to bar the Justice Department from using federal money to interfere in state medical cannabis programs.

But Spicer took a harsh view of recreational marijuana.

"When you see something like the opioid addiction crisis blossoming in so many states around this country, the last thing we need to be doing is encouraging people. There is still a federal law we need to abide by," Spicer said.

Marijuana legalization advocates say if the Trump Justice Department makes big changes, it will be in for a fight.

"If Jeff Sessions is looking for ways to make the Justice Department even more unpopular than it is right now, then you know go ahead and pick that battle over marijuana," says Michael Collins, deputy director at the Drug Policy Alliance. "But it's going to be a very unpopular fight and it's going to be something that he receives a lot of blowback for."

Collins says more than 60 percent of Americans live in places where marijuana is legal in some form or other.

Those efforts, he says, will continue in two years and four years when President Trump could appear on the ballot right next to marijuana legalization efforts in many more states.