When I started answering about the question, the interviewer didn't bother to listen he said I want Platform agnostic answers or answers only based on open source spark. When I tried answering him he quickly changed his stance and jumped to HDI and EMR distributions and started talking about partitions and cores. The person had no clue what he wanted to ask only focused on his side. I am sure its a good company but the interviewer should be clear what he is looking for and what are his questions.