There are at least two facets to Adam’s responsibility. His responsibility toward God is to see, as far as possible, that God’s will is obeyed. Adam’s responsibility toward his family is to provide guidance and protection. He fails in both respects.
Since God holds Adam especially responsible, what should Adam have done? How could he have averted the disaster of sin? He should have been purposeful about his role. The person to be held responsible cannot afford to sit back and watch how matters develop on their own. Fulfillment of responsibility seldom happens by accident. It usually requires the responsible person to lead.
Instead of letting the snake direct thinking about the fruit, Adam should have guided thinking in God’s way. Instead of following his wife’s example, Adam should have been setting the example. His eating of the fruit with her is not just a personal sin. It is also a failure in his role. As the first created and the first warned, he is primarily responsible to lead toward obedience and safety.