That wasn't the only thorny issue Obama faced in his trip.
The host of the Summit of the Americas, Colombian President Juan Manuel Santos, is considered one of Washington's closest allies. But speaking to an overflowing banquet hall of CEOs and presidents, Santos said it was perhaps time for a change in drug policy — perhaps a big change.
He meant possibly veering from the U.S.'s tough military approach against drug gangs in the region.
"After an objective, rigorous analysis of policy," Santos said, "we might reach another conclusion on strategy. That's what we're saying."
Options that some leaders have proposed include decriminalizing some drugs.
In response, Obama said he welcomes a debate, but he was clear where the U.S. stands in the war on drugs.
"I, personally, and my administration's position, is that legalization is not the answer," he said.
Celebrity And Scandal
On the surface, the summit in Cartagena on the Caribbean was all about regional unity, future opportunities and new-found prosperity in countries that had once been unstable and poor.
It was all presented with a dash of show biz.
Colombian pop star Shakira sang her country's national anthem for the 30 heads of state. Another popular singer, Carlos Vives, entertained hundreds of summit guests with his brand of folkloric rock.
Not everything went according to script.
The Secret Service placed 11 agents on administrative leave while investigating allegations that the men had brought prostitutes into their hotel rooms. The men, who'd been in Cartagena before the president's arrival, where sent home and replaced.
Officials said Obama's security was never compromised, but the incident did overshadow the summit.
Debating U.S. Policies
The region's toughest disputes — on drugs, Cuba and monetary policy — were mostly debated behind closed doors.
Still, at a forum on Saturday, the underlying tensions between some Latin American countries and the region's superpower made it to the surface.
That was where Obama, Santos and Brazil's president, Dilma Rousseff, debated the issues. The moderator was Hardball's Chris Matthews.
Rousseff was most direct, saying that what she calls the expansionist monetary policy of developed countries is creating a monetary tsunami in hers.
She argues that the U.S.'s low interest rates and spending cuts drive up Brazil's currency. That makes it more expensive for Brazilian exporters and floods Brazil with imports.
Obama gently defended the U.S. and instead said he wanted to focus on the future.
"We've never been more excited about the prospects of working as equal partners with our brothers and sisters in Latin America and the Caribbean," he said, "because that's going to be the key to our success."