Atari originally licensed the 6502-assembler code for Microsoft BASIC. This came in two versions, a 9k one and a more widely used 12k version. The difference between the two was in the math library, the smaller version used 6-byte numbers while the larger used 9-byte numbers. The Atari hardware design limited cartridges to 8k, so the Atari programmers started with the smaller version and struggled to cut it down enough to fit in a cart. That was bad enough of its own, but they really wanted to add additional instructions to take advantage of the Atari's graphics and sound as well. Eventually they gave up and went looking for a 3rd party to do it for them.
Shepardson Microsystems Inc, or SMI for short, won the contract. They proposed a simplified syntax and the cutting of a number of rarely-used features, leaving more room for new commands for graphics and sound. Even then, the result required about 10k, so to cross the remaining gap to 8k, some of the core libraries were moved out of the language and into the operating system ROMs. This had the side-effect of allowing any other language on the Atari to use these routines as well.
By the time SMI was hired, Atari was in something of a rush to get the BASIC. They were planning to show the machines in January 1979, and the contract required SMI to deliver the final version by April of that year. SMI completed it well in advance of the deadline, in October 1978, so Atari took an early version of the code with them to the CES show in January. To SMI's surprise, they learned that Atari had begun burning that version to ROM, even though it had several known bugs. SMI offered an updated version with various fixes, but Atari didn't bother using it, and instead shipped the buggy version for years.
Two major pieces of code were moved out of the BASIC into the OS ROM. The first was a set of graphics routines to set up the screen, draw lines, and similar. The second was the floating point math system, based on a new implementation of the 6-byte binary-coded-decimal (BCD) format. Both libraries were notoriously slow. Generally, Atari BASIC was among the slowest BASICs of its era, both due to the OS code and two problems involving loops.
The performance issues led to a profusion of 3rd party BASICs, some of which continue to be developed to this day. By replacing the math libraries and fixing these two loop issues, speed improves on the order of 3 to 5 times in most programs, and this is a common feature of 3rd party BASICs like TURBO-BASIC XL and Altirra Basic. For even higher performance, FastBasic uses a p-code system that can be quickly interpreted.
Most BASICs of the era had the concept of "immediate mode" and "program mode", and some commands could only be used in one more or the other. A good example in MS BASIC is the LIST command, which could only be used in immediate mode, at the "command line". Atari BASIC removed this limitation, one could write a program containing 10 LIST.
Additionally, most BASICs had commands that read input or produced output, for example, the INPUT can be thought of as the opposite of PRINT. Atari BASIC was orthogonal in that every such command had a partner. For instance, LIST produces output, so the new command ENTER reversed this, reading a program listing from a device. This opened up a number of possible overlay techniques that other BASICs lacked.
The most noticable difference between Atari BASIC and MS-derived versions is the string handling. Atari BASIC used a greatly simplified system of character-arrays instead of the dynamic strings in MS. This meant that all strings had to be predefined using DIM, and their length could not be changed during run-time. There are a number of advantages to this approach, notably speed, but memory handling is more difficult and conversion of standard programs from MS listings is more difficult.
Most 3rd party BASICs add many of these features, and more.