As President Barack Obama marks his sixth Independence Day in the White House, his approval ratings remain near the lowest of his presidency.

Just 41 percent of Americans approve of the job the president's doing in office, according to a new CNN Poll of Polls. But while Obama's numbers are anything but impressive, the approval rating for Congress is significantly lower.

The CNN Poll of Polls -- compiled and released Thursday -- also indicates that 54 percent of the public gives Obama a thumbs down on his job performance. The poll averages the seven nonpartisan, live operator, national surveys of the president's approval rating conducted since the beginning of June.

Obama's approval rating -- a key indicator of a president's standing with Americans and of his clout here in the nation's capital -- is hovering slightly above where it stood in November and early December when it was at or near all-time lows in many national surveys. And it's some 10 percentage points below where it stood at the beginning of 2013 -- soon after his 2012 re-election victory.

The president's numbers tumbled after late spring and summer following controversies over the Edward Snowden intelligence leaks and congressional investigations into IRS targeting of conservative political groups. Then came October and the politically charged botched rollout of the website for the Affordable Care Act, his signature domestic policy achievement.

Coupled with legislative setbacks, many pundits labeled 2013 the worst year of Obama's presidency. And for the first time since taking over at the White House in 2009, a majority of the public surveyed disapproved of his job performance.

Obama's numbers slightly rebounded earlier this year before edging down to their current standing in the low 40s.

The release of the new CNN Poll of Poll comes as the scandal rocking the Department of Veterans Affairs has dominated recent headlines. And the president has also recently faced numerous international challenges, including the increased bloodshed in Iraq, the ongoing civil war in Syria, and the fighting in Ukraine, as well as the controversial swapping of five Taliban prisoners held at Guantanamo Bay for the release of a U.S. soldier held captive in Afghanistan.

Independent voters may be fueling the deterioration of Obama's approval ratings.

"The president is still riding high with Democrats, and has never been very popular among Republicans. But it's the independents who are dragging down Obama's ratings. In some polls only about a third of independents approve of his job performance," said CNN polling director Keating Holland.

How Obama compares to predecessors

So how does Obama stack up against his most immediate two-term predecessors as they marked their sixth Independence Day in the White House?

George W. Bush stood at 36 percent in 2006, Bill Clinton was at 60 percent in 1998. And Ronald Reagan was at a lofty 64 percent in 1986. Richard Nixon, just weeks before he resigned over the Watergate scandal, stood at 28 percent.

The CNN Poll of Polls release came one day after a Quinnipiac University survey indicated that Obama tops the list of the worst presidents since World War II. Thirty-three percent of people questioned in the poll said that Obama is the worst president since the Second World War, with 28 percent saying Bush was the worst. Thirteen percent picked Nixon, while 8 percent named Jimmy Carter.

It's important to note that Obama and Bush, his predecessor in the White House, are more in the public conscious than previous presidents, and that both have governed during a time of bitter partisanship that's made compromise between the two major political parties extremely difficult. When Quinnipiac last asked the question, in 2006, Bush topped the list, with 34 percent saying he was the nation's worst president.

Congressional approval abysmal

The presidential approval rating -- during a midterm election cycle -- is considered a key indicator of how the president's political party may fare on Election Day.

The silver lining for the president: While his numbers are nothing to brag about, the approval rating for Congress is in the gutter. Only 14 percent of Americans approve of the job federal lawmakers on Capitol Hill are doing, according to the new CNN Poll of Polls, with nearly eight in 10 giving Congress a thumbs down.

The 14 percent approval rating is slightly better than November, when just 10 percent of the public gave Congress a thumbs up. But it's down from a year ago, when the congressional approval average stood at 17 percent.

"It's easy for Americans to dislike Congress because most of them are familiar, at most, with only three members -- their two U.S. Senators and the U.S. Representative from their district. That leaves 532 strangers who are easy to despise," Holland added.

Control of Congress is at stake in November's midterm elections.

Democrats have a 55-45 majority in the Senate (53 Democrats and two independents who caucus with the party). But in the midterms, the party is defending 21 of the 36 seats up for grabs, with half of those Democratic-held seats in red or purple states. In the House, the Democrats need to pick up a challenging 17 Republican-held seats to win back the majority from the GOP.

The CNN Poll of Polls is an average of the seven nonpartisan, live operator, national surveys of the president's approval rating conducted since the beginning of June: Gallup daily tracking poll (June 30-July 2); Quinnipiac University (June 24-30); Fox News (June 21-23); CBS News/New York Times (June 20-22); American Research Group (June 17-20); NBC News/Wall Street Journal (June 11-15) and Bloomberg (June 6-9).

The CNN Poll of Polls on Congress is an average of three nonpartisan, live operator, national surveys of congressional approval conducted since the start of June: CBS News/New York Times (June 20-22); Gallup (June 5-8); and Fox News (June 1-3).

Since it is an average of multiple surveys, the Poll of Polls does not have a sampling error.