Binary variables have only two values. For example the variable Gender may have the values male or female. These values are usually coded as '0' and '1' when data is entered to SPSS. Binary variables are the outcome variable of logistic regression.
Bivariate data or tests refer to situations where the relationship between two variables is being examined. For example, bivariate correlation refers to the correlation between two variables.