Tokens in C++

 Tokens in C++:-

In a passage of a text, individual words and punctuation marks are called as tokens. Tokens act as building blocks of a program. Tokens in C++ are referred to as the smallest individual units in a program.  C++ programs written using these tokens and syntax of the language. These tokens are the basic entities of the language. Tokens are classified into six categories.

1. Keywords

2. Identifiers

3. Constants

4. Strings

5. Operators

6. Special Symbols

Keywords:

          Keywords are reserved words which have fixed meaning, and its meaning cannot be changed. Each keyword is meant to perform a specific function in a program. The meaning and working of these keywords are already known to the compiler. It is important to note that we cannot use C++ keywords for giving variable names.

alignas	alignof	asm	auto	bool	break
case	catch	char	char16_t	char32_t	class
const	constexpr	const_cast	continue	decltype	default
delete	double	do	dynamic_cast	else	enum
explicit	export	extern	FALSE	float	for
friend	goto	if	inline	int	long
mutable	namespace	new	noexcept	nullptr	operator
private	protected	public	register	reinterpret_cast	return
short	signed	sizeof	static	static_assert	static_cast
struct	switch	template	this	thread_local	throw
TRUE	try	typedef	typeid	typename	union
unsigned	using	virtual	void	volatile	wchar_t
while	–	–	–	–	–

Identifiers:

               Identifiers refer to the names of variables, functions and arrays. These are user-defined names and consist of a sequence of letters and digits, with a letter as a first character. Both uppercase and lowercase letters are permitted, although lowercase letters are commonly used. The underscore character is also permitted in identifiers. It is usually used as a link between two words in long identifiers.

Rules for Identifiers:-

1.      First character must be an alphabet or underscore.

2.      Must consist only letters, digits & underscore.

3.      Only first 32 characters are significant.

4.      Cannot us a keyword.

5.      Must not contain a white space or special symbol.

Constants:-

            Constants in C++ refer to fixed values that do not change during the execution of a program. A constant is used to describe a numerical value, or a character string. Numerical constants may be expressed as real constants, integer constants. The constants are treated just like regular variables except that their values cannot be modified after their definition.

Strings:-

                 Strings in C++ are used to store letters and digits. Strings can be referred to as an array of characters as well as an individual data type. A string is a sequence of characters enclosed within the double quotes. The termination of a string in C++ is represented by the null character, that is, ‘\0’. The size of a string is the number of individual characters it has. E.g. “Shardul Patil”, “11+30”, “Hello!”, “1997”, etc.

Operator:

          An operator is a symbol that tells the computer to perform certain mathematical or logical manipulation. Operators are used in program to manipulate data and variables. C++ operators can be classified into several categories as:

1.      Arithmetic Operator

2.      Relational Operator

3.      Logical Operator

4.      Assignment Operator

5.      Increment and Decrement Operator

6.      Conditional Operator

7.      Bitwise Operator

8.      Special Operator

Special Symbol:

    Apart from letters and digits and operators, there are some special characters in C++ which help you manipulate or perform data operations. Each special symbol has a specific meaning to the C++ compiler. E.g. [], ;, #, etc.