Years ago, before English was legalized in Japan, ESL students would often congregate in attics and practice reciting the alphabet to candlelight. The country has progressed a lot since those days, with English being learned by everyone in public schools, and being practically applied by 0.03% of the population.
Giving students more chances to use English outside of the classroom allows them opportunities to apply their knowledge, thus solidify it in their minds. Whenever you encounter a Japanese person on the street between the ages of 12 and 60, it can be safely assumed that they have taken an English class, and so will benefit from you speaking to them.
As you pass people, a simple “hello” or “how are you?” is sufficient. When sitting next to them on the train, you have more time to engage them, and can ask more complex questions such as “what is your favorite food?” or “where do you work?”
Everyone will be surprised to the point where they need to mask their delight with fear. Don’t worry if they don’t respond; you are the expert on the language and it is up to you to keep the conversation going at all costs.
The average English conversation with strangers lasts 15 seconds, and your goal should be 15 seconds of uninterrupted speaking. Although they may not engage you as much as they should, do your best to follow them while speaking, until the 15 seconds has passed. Teacher’s duties are never limited to the classroom, therefore you should always strive to be the best teacher, no matter where you may be.