What should I do with my time during the Covid-19 pandemic?  Having realised I am hopeless at DIY tasks and that I am annoying my nearest and dearest; I decide to retreat to a place of safety and camaraderie.  ResearchED LOOM Durrington may not look like previous conferences, but once inside it is a comfy pair of slippers.  29 top class sessions, many from the family favourite range, thank you to Shaun Allison for pulling it all together.

What it lacks in shiny new school buildings, it makes up with a smorgasbord of content with the added benefit of not having to queue outside rooms to ensure you get in.   The usual ResearchEd service continues like a Subway sandwich, personalised and made to order.  You really can have it your way.  We are locked down but not locked out and in the words of Brenda, we will meet again.

My route through the day.

1. Session 1: Dylan Wiliam – Teacher Quality: What is is, Why it Matters, How to get more of it.
2. Session 2: Paul Kirschner – Tips for effective teaching if you have to teach at a distance.
3. Session 3: Harry Fletcher-Wood. How can we get students to ‘turn up’ in remote learning?
4. Session 4: Stephen Lane. The Three Cs of Remote Pastoral Care.
5. Session 5: Rob Coe, Stuart Klime – Assessment in Distance Learning
6. Session 6: Stuart Klime. Using MCQs.
7. Session 7: Phil Stock: We don’t deserve nice things: how good ideas get ruined and what we can do about it.

Opening Address.

A lovely opening by Tom Bennett, we live in a time when research evidence for what we do next will become even more important.  There is no playbook for educationalists but the ResearchEd questions have never been more pertinent.

• What does work?
• What works when?
• Who does it work for?
• Who does it not work for?

There is an emerging field of research about technology and learning but it is not a settled science.  Perhaps by the time we return to normal, we will be able to integrate some of the best elements into our normal practice.

Pro-tip: On Loom you can listen to the talks at a quicker speed which adds a comical dimension to your online CPD.

https://youtu.be/StWd89fzAHM

Session 1.  Dylan WIliam: Teacher Quality: What is it, why it matters and how to get more of it.

Teacher quality (researchED 2020) — Watch Video

How do we assess teacher quality?  There are four main strands to the research evidence: quality of curriculum, time to plan, resources available and skills of the teacher.  Wiliam wants to deal with the last strand, skills of the teacher.

There is a positive correlation between the skills of a teacher and student progress.  The most effective teachers are 400% more effective and can teach the same material in 6 months, that will take an average teacher 1 year and a less effective teacher 2 years.  The most effective teachers will have no difference in the progress of their advantaged and disadvantaged pupils.

Can we identify good teachers?

1. Can we identify good teachers from observation?

We are not good at identifying good teachers.  Studies suggest most people, even experienced school leaders cannot make reliable observations of teaching practice.  (To achieve 90% reliability in these observations, we would need 30 observations by 5 different observers.)

If you are teaching the top set you are X6 more likely to be given an outstanding rating than if you teach the bottom set.  Everyone looks better when teaching the smart students.

1. Can we identify good teachers from their qualifications?

There is no significant relationship between teacher qualifications and good teaching.  In fact, in maths there is an interesting inverse relationship between maths qualification and effectiveness in teaching mathematics (Harris & Sass, 2007).

2. Can we identify good teachers after training?

Danielson (1996) suggests 4 domains of professional practice:

• Planning and preparation.
• Classroom environment.
• Instruction.
• Professional responsibilities.

1&4 have no impact on student achievement but 2&3 seem to have some impact.

3. Can we identify good teachers from the student test scores before and after the teaching?

Carrell & West (2010) study of US Military Instructors, random allocation to instructors.  Less experienced instructors had higher end of session test scores but lower follow on test scores.

Some teachers may teach to the test and look better in the short term but not be effective for the long term (Dunning-Kruger Effect).

4. Combining sources.

Gates Foundation (2012) tried to combine value-added scores (81%), classroom observation (17%) and student perception (2%).  You would need to collect data for each teacher for 9 YEARS for it to be 90% reliable.

“We should not try to evaluate teachers, we cannot do it accurately, we have better things to do with our time”

These conclusions mean it is difficult to get rid of least effective teachers as we do not really know who they are.  Some teachers are good in the short term but not effective for long term student outcomes and vice versa.

Every teacher builds on the foundations of previous teachers. A good teacher benefits a student’s learning for 3 YEARS after their teaching.

Take-away.

The “Love the one you are with” strategy.

Effort spent trying to evaluate teachers is rarely effective, instead we should focus on trying to improve teachers.  We should get rid of teachers who say they do not need to get better, those who blame the students for their failures.  We need to apply Dweck’s work to the teaching profession, what else can I try to improve the learning for my students?

We need create a school culture where every single teacher in school believes they need to improve not because they are not good enough but because they can be better.

We cannot fire our way to Finland, if we want to improve we need to build a world class workforce from within.  Mastery takes time.  How good could our school be in 5 or 10 years time?

Session 2.  Paul Kirschner – Tips for effective teaching if you have to teach at a distance.

Paul Kirschner – Tips for effective teaching if you have to teach at a distance — Watch Video
Based on book – Lessons for Learning.

This is not rocket science.  Online platforms offer some solutions but the instructional techniques are not the same as the ones we use in the classroom.

Tip 1. Stick to the essentials.

Beware of offering too much new subject matter.  Consider using the time to review what they have already learned.  Think of the Summer-dip.  If we do not do this what they have already learned will be forgotten.

Tip 2. Frame new subject material in a larger picture.

Provide them with anchor points to help them structure any new material and context.

Tip 3. Refer to relevant prior knowledge.

The most important factor in learning new things, is what one already knows.  Where can they go if they have forgotten previous knowledge and need to look it up?

Tip 4. Communicate concrete goals and success criteria.

Obviously, these are clear to you but often not clear to the students.  To what level should students explore this topic?  How much depth and detail?  How will they know?

Tip 5. Use examples.

Have students study a worked-out example or model before new content.  Then you can remove the steps one at a time.

Tip 6. Offer students support during practice.

Online or scaffolded support.

Tip 7.  Actively processing the material.

Studying a topic is not enough, they need to elaborate, expand and ask questions.  Formulate epistemic questions; who, what, when, how?  Get them thinking.

Tip 8. How will they know if they know it?

Can they take a practice or retrieval test.

Tip 9. Provide feedback.

Provide them on their answers.

Tip 10. Spread the learning out over time.

Spacing effect, short bursts of practice and learning over time.

Take away

Do not do what you normally do.  Short assignments where they can show you what they have learned.  Use online stuff that is already made, do not re-invent the wheel.

 Session 3.  Harry Fletcher-Wood.  How can we get students to ‘turn up’ in remote learning?

 How can we get students to ‘turn up’ to remote learning? — Watch Video

Blogpost – https://improvingteaching.co.uk/2020/03/19/motivating-distant-learners-schools-under-coronavirus/

Early evidence only 1/3 students handing stuff in or doing the homework.  Those who are least likely to participate are the ones we are most worried about.

1. Specifying goals

What looks like resistance is a lack of clarity.  They are not pushing back they just where not sure what they had to do?  Create a clear habit, check in, complete the tasks?  Students need to get into a routine.

2. Motivating action.

Most powerful influence is that we tend to do what other people do when we are uncertain.

“We expect every student to …”

“Almost everyone completed …”

“More students than ever completed …”

Create a sense in students minds about what others are doing.

3. Planning action.

Help students to plan their time.  Planning where and when you will do your work makes it more likely that they will do it.

4. Relaunching habits.

It takes a long time to form a habit.  Average basic habits 66 days, going to the gym x4 a week for 6 weeks.  Our students have only been doing this for a week, it will take time for them to establish a routine.  We need to continue providing support and helping them to reset and relaunch their habits.  Do not despair if they are not doing it yet.

Take away.

We are only a few weeks into this new norm.  It will take a while; we need to support students in establishing the new norms.

Session 4: Stephen Lane.  The Three Cs of Remote Pastoral Care.

Is there a gap in the research evidence about pastoral care?  

1. Contact

How do we maintain contact with vulnerable children?  Are we in regular contact with the children and families?   What about those who are not defined as vulnerable?  How do we maintain contact with them?   For those on the margins, the closure of schools shuts down a vast network of peer and adult support for all our students.

It is important we maintain contact and that students feel their school still cares about them.  We need to be mindful of the impact of isolation in adolescents.  Home networks may not be as supportive as we would like, and for some their parents may not be the people they speak to for support.

Study of impact of Hurricane Katrina, the isolation and trauma had a significant impact on young people’s mental health / anomie.

How do we know that young people are attending to their well-being?  How do we make sure we are in contact with all of our students?  They may not check their emails.

CAMHS and social services will be stretched.  Can we use phone calls and home visits?

2. Congregation / Community.

Pastoral care has its roots church / parish practice.   School communities act like a congregation / community.  Who acts in the role as ‘the chaplain’ and thinking about the welfare of the students beyond their academic progress?  The urgent focus has been on the Y11 and Y13 students and their transition, but next we need to consider their emotional well-being.

What emotional support do we have in place? This is weird for us as adults, how strange must it be for teenagers.

How will we listen to the concerns of the students?  We are focusing on the stuff we are sending out and the academic feedback, but what have we done to open a channel of communication back to pastoral leaders.

How do we give them a sense that things will be okay and that there will be a future beyond the next few weeks?  How will we help those who lose loved ones?

How will we communicate our pastoral messages to students?  We often do not apply the same joined up thinking to our pastoral curriculum

3. Curriculum

What pastoral knowledge do we need them to learn?  What is needed from the PSHE, Citizenship or Character agendas.

How can pastoral curriculum be knowledge-rich?

We should not neglect the personal development part of the curriculum.  What is the learning that needs to happen about keeping safe and being a good citizen on an online curriculum?  How can they alert people to any problems they are experiencing with online media.

Take Away.

Think about how we listen to student concerns.

Develop online personal development curriculum.

How do pastoral leaders support their colleagues?

Session 5:  Rob Coe, Stuart Klime – Assessment in Distance Learning

Rob Coe: Assessment in Distance Learning. researchEd 4 April 2020 — Watch Video

Distance learning and assessment is hard.  When you are not in the classroom with students it is even more difficult to know what is in a student’s head.  However, we know how to make assessment work.

1. Retrieval Practice and the testing effect.

Lots of evidence that testing benefits learning.

• Testing effect works for both low-level factual recall AND more complex learning, understanding and application.
• Retrieval practice helps recall after a delay.
• Making errors in retrieval practice is not a bad thing, the important factor is the feedback.
• Attempts to retrieve can help learning even if they are unsuccessful.
• Generation of answers (free response) works better than just recognition (multiple-choice).

How and why does retrieval work?

• Strengthens the cues that enable retrieval of previously learnt material. In struggling to remember something that is almost forgotten, students search for connections that may help bring it to mind.  Retrieval works best when we search for something once well connected but almost forgotten
• Act of successfully retrieving leaves a memory trace or path that makes it easier to follow again.
• Act of retrieval generates elaboration which creates and strengthens connections between ideas – making them better understood. Having to think hard makes u connect and explain.
• Generating answers requires a consolidation and elaboration of thinking that secures a better understanding. It is only when I try to explain it that I can fully make sense of it.
• Difficult or unsuccessful attempts at retrieval direct attention and increase salience of the target memory. Retrieval works best when it makes students realise they have forgotten stuff they need to know.

Making it work in regular classrooms. 

Reasons it might not work.

1. Teachers might ask students to retrieve information they had not properly understood or assimilated yet.
2. Teachers might generate retrieval questions that focus too narrowly on factual recall rather than higher-order thinking.
3. Quiz questions might be too easy. Teachers use them to confirm and boost confidence rather than challenge.  OR just misjudge the level.
4. Teachers find the pressure from curriculum coverage or just the effort of compiling quizzes too much and end up not doing it.
5. Teachers allocate too much time to quizzes, effectively losing all their teaching time to cover new material.
6. Teachers ask retrieval questions about material that the students have not had time to forget. Retrieval works best when it is effortful and reinforces f added memories: too soon and the benefit is lost.

Retrieval is a powerful strategy with a lot of evidence to support it, but it is quite complex to get right.

2. Feedback: mechanisms and principles.

Research suggests feedback is one of the most powerful strategies we have.

• Feedback clarifies and draws attention to goals which makes them more salient (prioritise different goals, task / process / metacognitive / self-worth).
• Feedback draws attention to the gap between actual and desired levels of performance (how am I doing?)
• Feedback can indicate productive next steps.
• Feedback can cue attributions for success or failure.

Practical principles of good feedback.

1. Feedback should aim to improve the next time the student does a similar task not this one. Should make the learner better at doing the task rather than improving this task.
2. When you give feedback, get feedback on the feedback. Check they actually follow the advice.  Often students are not sure what to do to get better.  Giving feedback on their ability to follow advice. Feedback is not a substitute for instruction.  If they have not understood the idea, it needs re-teaching.
3. Good feedback (guidance) hits the sweet spot between challenge competence: don’t help too much.
4. Feedback is not just about task-performance: also motivation and self-regulation; You can do better!

Take Away

Retrieval is a powerful strategy with a lot of evidence to support it but it is quite complex to get right.

 Use principles of good feedback in distance learning. 

Session 6: Stuart Klime. Using MCQs.

Practical tips.

Use letters, distractors and vertical letters for ease of use.

• Aligned with the curriculum
• Examine important not trivial content.
• Avoid unnecessary hard vocabulary, should be at appropriate level.
• Avoid gender and cultural assumptions.
• Do not try and catch people out or opinion-based questions.
• Stem should be meaningful – avoid choose the statement which is true?
• Stem should not include irrelevant material (it distracts them).
• Stem should not be negatively phrased. … which is not an example.
• The stem should be a question.
• Avoid fill in the blanks or a statement.
• Alternatives should be similar in content.
• Alternatives should be mutually exclusive and not overlap.
• Avoid greater detail in the correct option, this gives a clue to the correct answer.
• Don’t bother with none of the above.

Further resources.

Evidence Based Education Resource Library.

https://evidencebased.education

Take Away

It takes time to develop and create good multiple-choice questions.  The value they give you back is worth the effort, you get common misconceptions.  Make the distractors work for you, three plausible alternatives.   This is a good task for collaboration with colleagues.

Session 7: Phil Stock:  We don’t deserve nice things: how good ideas get ruined and what we can do about it. 

Online meetings, lessons can produce a lot of misunderstandings.  Phil wonders if that has been happening in education as well.

This has been happening is education for a long time.  We know a lot about how to organise learning and improve outcomes for young people.  Are we seeing success in outcomes from the move towards evidence-based practice?

There is a great deal of evidence but not necessarily mobilising into successful outcomes.

1. Misalignment of values and beliefs between those producing the evidence and those having to implement it.
2. Timing – good ideas work best in the right moments.
3. Resources – we do not always have the infrastructure to implement ideas.
4. Misunderstanding in the nuance behind the practical.

Why are we not seeing any impact?

Short term demands from many stakeholders lead to continual demands to make an immediate difference.  Is it possible to embed the changes for short term gains?

We are awash with many great ideas about how to improve teaching and learning.  There is not a failure of knowledge but a failure of implementation.  We need to be more critical about the implementation of new ideas.

For example, working memory and cognitive load has become potentially corrupted and misunderstood.  It has been adopted by Ofsted, who have a new interest in the ‘learning sciences’.

Learning sciences (Bjork – desirable difficulties, Sweller – cognitive load theory, Dual coding).

The way these ideas have been operationalised:

1. Teach for long term memory and understanding (how we organise the curriculum)
2. Powerful knowledge curriculum (carefully sequenced curriculum).
3. Highly focused students and learning hard content.

 

Three tropes that are how schools understand cognitive science and the concepts of working memory.Knowledge organisers – the rush to put them together shows us how far they have strayed from their original idea.  Interleaved curriculum – many schools have changed KS3 curriculum to interleave different topics or interweave. Curriculum road map – draws on dual coding and sequencing.  What was wrong with laying it out in a table.  It does not demonstrate the progression clearly.

Cognitive science is important but alongside EEF Toolkit of classroom experiments and craft expertise.

We need to consider how we implement ideas.  We cannot just borrow ideas from others without fully understanding the evidence.

 Take Away

It’s all about implementation. 

1. Be diagnostic. What problem is this change looking to address.  Will it undo existing achievements?  What changes will we see?  What does  improvement look like?
2. Be informed. Understand research methods and a wide evidence base.  Seek out critical voices and those who challenge our thinking.  EEF Teacher choice trails, which is the best way to start the lesson quizzing Vs discussion?  Many of the studies that prove this are laboratory conditions.  Do decades of findings in a laboratory transfer to classroom settings.  IEE Engaging with Evidence Guide (2019) Red Flags – how to be critical of research and website called That’s A Claim.
3. Be strategic. Clear plan of what you want to achieve, what training will it require?  What are the opportunity costs of this new change?  If we do more quizzes what is lost?
4. Be evaluative. Gathering feedback on the ground from staff and students.  Analysing impact on results and consider the undesirable consequences.
5. Be honest. Sunk cost fallacy – just because you have put a lot of time into something, does not mean it is the right thing.  Confirmation bias – do we look for evidence which supports our much loved idea.

Further resources

EEF Implementation Guidance Report.

A simple formula for making change. 

Success = diagnostic, informed, strategic, evaluative and honest.

 My summary.  

Phew, what an enjoyable couple of hours.  Reassuring to hear to many speakers articulate what I am thinking during this crises.  We need to hold on to the gains we have made with using research evidence to help us work out the next bit.