Readings: Before you get started, it will be useful if you download a copy of Herrington et al (2007).
Setting the scene
Before leaping into design-based research, let’s just set the scene in terms of where you are, and where you are going.
Where are you now?
My hope is that by now you’ve gained some experience and insights into NGL and formed some conclusions about what may or may not work for you and the folk for whom you are “teacher”. As part of this I am assuming that you have completed Assignment 1 and in particular the “How NGL can inform my role as teacher” post.
Important: I really do hope that you understand the NGL can and generally should mean much more than simply having students use blogs, twitter, and Feedly within existing learning environments. These are reasonable tools for working toward NGL principles and goals, however, simply using those tools doesn’t mean you’re engaged in NGL. In addition, NGL can be a paradigm shift. A gateway to a very different way of thinking about knowing, learning and teaching.
Remember the Kligyte (2009) reading from week 2 and its argument that network learning changes how we “create, analyse, and share knowledge” (Kligyte, 2099, p. 540). Arguably, the fundamentals of learning and teaching.
Question: As you work on your Assignment 2, keep asking yourself just how much your planned intervention is based on changes in how knowledge is created, analysed, and shared?
Where are you going?
In short, over the remaining weeks of semester you are going to engage in some serious, theoretically informed planning for how you might transform your teaching with NGL. That’s what assignment 2 is all about. This week will introduce you to design-based research, the method by which you will develop your plan. The rest of the semester involves you engaging in design-based research and developing your plan (Assignment 2).
What does “transform teaching” mean?
As with much in this course the concept “transform teaching” (and its component parts “transform” and “teaching”) is open to some serious negotiation and interpretation. The following points are intended to give you a taste of some (but not all) of the potential interpretations you might apply.
In terms of transformation, you might find it useful to return to the week 6 material. Especially the SAMR and RAT frameworks. In terms of SAMR, the use of transform should be interpreted as aiming towards Modification (a significant task redesign) and Redefinition (creation of new, previously inconceivable tasks).
The RAT framework can also be useful. In terms of the RAT framework, you should be aiming for transform (allows for forms of instruction and learning that were previously inconceivable). Beyond this, the RAT framework also identifies learning, instruction, and curriculum (what’s being learned) as areas where transformation can occur.
Question: As you develop Assignment 2, ask yourself at what level of SAMR/RAT is your intervention placed? What is being changed: learning, instruction, and/or curriculum?
What is “teaching”?
In terms of teaching there are a range of possible implications and understanding.
- Teaching isn’t limited to normal classroom or formal teaching.Remember the interpretation from the assessment page “‘teaching’ is interpreted as you doing something that helps others learn”. It might be a formal course or class, it might be one-on-one tutoring, or it might be helping other staff successfully complete a task.
- You don’t have to limit yourself to what you’ve done in Assignment 1.While the intent is that assignment 1’s “How NGL can inform my role as teacher” will feed into assignment 2, you don’t need to feel constrained by that. If you’ve had some brilliant insight since completing assignment 1 go with it.
- You don’t have to limit yourself to actual teaching.”Teaching” can mean anything and everything that helps you teach. Actual instruction is a (very important) part of this. Other tasks might include professional development, assessment, evaluation, reflection, modification of the “classroom” environment etc.
- You don’t have to limit yourself to just one thing.There’s no need for Assignment 2 to focus on one massive intervention such as “all my students will set up individual blogs and use feedly” (which, by the way, is a pretty bad description/focus). It can be a combination of multiple different interventions that are mutually supportive.
- You do NOT have to implement your idea.Assignment 2 is about formulating the plan. There is no need to implement this plan for this course, or ever. It is hoped that it would be something directly useful to your context that you would eventually implement, but there is no need now.
Way back in 2013 I identified a prime focus for my work as being to address institutional e-learning’s TPACK problem. For me there is a NGL-based solution to this problem. In my case, I’ve interpreted this role as “teaching teachers”. My premise is that university e-learning tends to be so bad, because the institutional context doesn’t have enough or the right type and mix of technological, pedagogical, and content knowledge. My aim “as teacher” is to help discover ways that institution’s can improve their TPACK knowledge.
The plan is to use this example as an illustration throughout the rest of this week’s material. This will draw on some recent publications to illustrate. The following lists publications that have flowed from that initial conceptualisation of the TPACK problem. The idea is NOT that you need to read each of these publications. Instead it’s to illustrate some of the different ways that NGL-based theory/principles can be used to understand and inform teaching practice.
Jones, D., & Clark, D. (2014). Breaking BAD to bridge the reality/rhetoric chasm. In B. Hegarty, J. McDonald, & S. Loke (Eds.), Rhetoric and Reality: Critical perspectives on educational technology. Proceedings ascilite Dunedin 2014 (pp. 262–272). Dunedin. Retrieved from http://ascilite2014.otago.ac.nz/files/fullpapers/221-Jones.pdf
Develops the BAD mindset as a set of principles for how institutional e-learning should be developed and illustrates how it helps complement existing institutional practices.
Jones, D., Heffernan, A., & Albion, P. R. (2015). TPACK as shared practice: Toward a research agenda. In D. Slykhuis & G. Marks (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2015 (pp. 3287–3294). Las Vegas, NV: AACE. Retrieved from http://www.editlib.org/p/150454/
Describes a distributed view of TPACK. The distributed aspect borrows heavily on aspects of NGL. That distributed view of TPACK is then used to analyse what three teacher educators have done to understand what they do to engage in e-learning.
Jones, D., Albion, P., & Heffernan, A. (2016). Mapping the digital practices of teacher educators: Implications for teacher education in changing digital landscapes. In Proceedings of Society for Information Technology & Teacher Education International Conference 2016 (pp. 2878–2886). Chesapeake, VA: Association for the Advancement of Computing in Education.
Building on the idea of protean digital technologies (from the TPACK as shared practice paper) this paper uses the Digital Visitors and Residents framework to understand how teacher educators modify the digital technologies they use in their teaching.
Getting to know design-based research (DBR)
Design-based research is a particular approach to research that seeks to address practical problems by using theories and other knowledge to develop and enhance new practices, tools and theories. The aim is to improve both practice and theory. I’m biased, but I believe it can be a useful way for teachers to think about their everyday practice. For assignment 2 you will focused on improving your teaching practice.
Reading: Look at this page gives for a brief summary of DBR. Note: Be sure to have Diigo available in your browser, I’ve annotated the page with a few comments related to the course. If you’re a member of the NGL Diigo group, you should be able to see those annotations.
Reading: Read the first section (titled “Design-based research and educational technology”) of Herrington et. al. (2007) (the reading you were asked to download at the start of this week). This section provides another brief background to DBR, how it fits with educational technology/research and a high level idea of the phases involved in DBR and how it compares to more traditional research.
Note: Herrington et al (2007) is aimed at helping doctoral students and supervisors engage in DBR. While you are engaged in a similar sort of research, you won’t be completing it at the same level, spending the same amount of time, or required to engage with all that is in Herrington et al (2007).
Phases of DBR and Assignment 2
When reading Herrington et al (2007), you should have seen a figure that included the following phases of DBR.
For Assignment 2, you will be focused on the first two phases:
- Analysing your teaching context and the problem(s) you see that NGL might usefully assist with, and then
- applying NGL ideas and principles to design (but NOT develop) some contextually appropriate way of addressing these problems.
i.e. you will not be expected to engage in the last two phases of DBR for Assignment 2.
Assignment 2 as a research proposal
Assignment 2 will take the form of a research proposal as described by Herrington et al (2007). Your “proposal” will not be as detailed (you’re doing a single assignment for a single Masters course, not a PhD) or contain all of the sections that Herrington et al (2007) discuss.
- the section titled “The research proposal” from Herrington et al (2007) that describes what is meant by a research proposal; and,
- the section titled “Guidelines on preparing the design-based research proposal”, but stop at the heading “Statement of problem”.
The following table is an adaptation of the table “PHASE 1” from Herrington et al (2007). It’s been modified to focus on the elements of a research proposal that you must included in your Assignment 2. It also highlights the connection between these requirements and the Assignment 2 specification and rubric.
The rest of this week’s material uses Herrington et al (2007) to explain each of the elements of a DBR research proposal. Use the links in the table below to learn more about what is required.
|Phase of design-based research (Reeves, 2006)||The topics/elements that need to be described|
|PHASE 1: Analysis of practical problems by researchers and practitioners in collaboration||
|PHASE 2: Development of solutions informed by existing design principles and technological innovations||
The DBR Planning Template
NEW and improved for 2016. Professor Jan Herrington – the lead author of Herrington et al (2007) – maintains a web page with a collection of DBR links and resources. One of the resources on that page is a Word document that contains a DBR planning template. The template is based on the four DBR phases proposed in Herrington et al (2007) and thus what is covered below.
The template appears to provide a useful scaffold for working and thinking about your DBR proposal and the following has been modified to include activities that suggest you make use of the template.
Structure for your assignment 2
As outlined in the above table and the assessment page, there are two parts to Assignment 2
- Peer review; and,
- Research proposal.
As outlined in the table above, the peer review part equates to the “Consultation with researchers and practitioners” element identified by Herrington et al (2007). The assessment page talks about the format for your peer review.
Recommendation (but not a requirement): My suggestion is that the research proposal you submit for Assignment 2 should use a structure informed by the above table with the addition of a three standard essay sections. i.e. I suggest (but do not require) that your proposal have the following sections
- Introduction – as per a normal academic essay.
- Statement of the problem
- Research questions
- Literature review
- Draft principles
- Proposed intervention
- Conclusions – as per a normal academic essay.
- References – as per a normal academic essay.
Statement of your problem
Reading: Read the section titled “Statement of problem” from Herrington et al (2007).
A key criteria for a DBR project is that it should “depart from a problem and then pursue both knowledge and interventions that address it” (McKenney & Reeves, 2013). i.e. there should be some sort of problem that sparks the intervention and research. In your proposal the statement of the problem section should
identify an issue or an opportunity, explore its history or background, and provide a convincing and persuasive argument that this problem is significant and worth researching. This includes articulating both the practical and scientific relevance of the study (Herrington et al, 2007)
It should draw on some literature to support it’s claims, but should not delve deeply into the literature.
While it’s not an exact match for what is required, the following excerpt from Jones and Clark (2014) gives a taste
In a newspaper article (Laxon, 2013) Professor Mark Brown makes the following comment on the quality of contemporary University e-learning:
E-learning’s a bit like teenage sex. Everyone says they’re doing it but not many people really are and those that are doing it are doing it very poorly. (n.p).
E-learning – defined by the OECD (2005) as the use of information and communications technology (ICT) to support and enhance learning and teaching – has been around for so long that there have been numerous debates about replacing it with other phrases. Regardless of the term used, there “has been a long-standing tendency in education for digital technologies to eventually fall short of the exaggerated expectations” (Selwyn, 2012, n.p.). Writing in the early 1990s Geoghagen (1994) seeks to understand why a three decade long “vision of a pedagogical utopia” (n.p.) promised by instructional technologies has failed to eventuate. Ten years on, Salmon (2005) notes that e-learning within universities is still struggling to move beyond projects driven by innovators and engage a significant percentage of students and staff. Even more recently, concerns remain about how much technology is being used to effectively enhance student learning (Kirkwood & Price, 2013). Given that “Australian universities have made very large investments in corporate educational technologies” (Holt et al., 2013, p. 388) it is increasingly important to understand and address the reality/rhetoric chasm around e-learning.
A previous participant in the course proposed the following set of questions to help her focus on her problem and possible solution
- What is the problem/challenge/focus?
- Why is it a problem?
- Who says, or, who agrees and doesn’t agree?
- What has been done so far to deal with this?
- Who tried it and what were their results?
- In light of all this, what else could be done, and what will be best for this particular problem?
- What makes this idea viable?
- What process of implementation will work best, and why?
- What is your problem? What are the problems in your teaching? I’m sure there are many. Spend some time (now and/or later) thinking about what those problems are and sharing them via your blog. Engage your PLNs and also draw on any relevant literature that might help identify relevant problems.
- Consider the set of questions above from a previous participant, especially the first 5 at this point.
- Download a copy of the Herrington DBR template and add in your ideas for question 1.1 – What is the educational problem that your research will address? This is something you will continue to work upon. Using a blog post (or more) to share you thinking around this question could be useful.
That last activity could be seen as the start of the consultation process.
Consultation with researchers and practitioners
Reading: Read the section titled “Consultation with researchers and practitioners” from Herrington et al (2007).
For assignment 2, you will be taking on the role of both researcher (you are the one trying to develop the principles behind an intervention) and practitioner (the intervention is being designed for your role “as teacher”). However, consultation and collaboration around DBR is a real advantage. As we don’t have the luxury (or perhaps constraint) of you joining “a robust research agenda” to get this consultation and collaboration another solution is required.
As it is currently written the peer review component of the assignment doesn’t quite capture the full detail of what is intended. It should not be interpreted as requiring you to get two friends to read your completed proposal (essay).
Instead, it should be interpreted as meaning you should be planning and implementing a variety of tactics to encourage a range of people to be commenting on, critiquing and offering suggestions on your assignment over its entire life cycle.
DBR, researchers, practitioners and you
DBR is typically explained as a researcher working with practitioners to solve a problem. For this assignment you are taking on both of these roles. You are a practitioner that is identifying a problem with your teaching practice. You are also the researcher who is developing a plan about how to use NGL principles to do something useful around that problem.
This double role makes the task more difficult. In particular, it makes it harder for you to really look at the nature of the problem from different perspectives. To avoid getting to caught up with your own schema. The peer review part of the this assignment is intended – amongst other things – to help deal with this difficulty.
Activity – the practitioners: Start working through section 1.2 Practitioners from the Herrington template. Answers to the questions in this section can help frame your planning for your peer review process.
Activity – the experts/researchers: The Herrington template focuses on talking with practitioners. While this is essential, it is also important that you broaden your ideas around potential solutions, especially NGL inspired solutions. Take the same questions from section 1.2 of the Herrington template and replace teachers/students/practitioners with experts and others. i.e. anyone else who has expertise – especially around NGL – that could help provide ideas and feedback on your thinking.
Remember, the peer review process for Assignment 2 is worth 10% of your result for this course.
Literature review and theoretical framework
Assuming you have worked through the above process, then you should be gaining some idea of the type of problem you are trying to explore/address using NGL principles. You may even have some ideas about how you will solve that problem. That’s inevitable, but is also something to be avoided. You don’t want to come to a solution just yet. The next step is to understand what work has already been done around the problem or problems that are similar. What are the ideas, insights and knowledge you can draw upon to better understand your problem and potential solutions?
Reading: Read the sections titled “Literature review” and “Theoretical framework” from Herrington et al (2007).
Your literature review
Your literature review will likely draw on both NGL literature, literature around your context “as teacher”, and other literature. In your literature review you are looking to achieve the types of aims outlined by Herrington et al (2007), including identifying:
- what is known about the problem you have identified.This is different from the statement of the problem. In the literature review you are trying to give a summary of the different perspectives of the problem. Trying to identify where your approach belongs, where it might borrow insight, or where it might be extending what is known a little. That last bit – extending – is necessary for a PhD, but not for your assignment 2.
- what is state-of-the-art in your context re: the problem and NGL.
- initial guidelines for how to go about the design of your intervention.
Activity: Complete section 1.3 of the Herrington DBR template, which asks you to identify what are the key references in your area of interest. In the context of this assignment, know would be a good time for you to identify the key NGL references that you might draw upon. In particular, it’s a good time to start refining the broad and ill-defined idea of NGL principles, into something specific which you can use.
This is where a theoretical framework can come in handy. A theoretical framework provides “the ‘lens’ through which the problem will be investigated, and it is also the place where the theoretical foundation of the proposed solution will be explained” (Herrington et al, 2007, p. 4095). The idea of NGL used in this course is specifically ill-defined to ensure that you feel free to draw upon any of the numerous available and related frameworks. Some of these have been introduced in the earlier weeks of this course.
Jones and Clark (2014) use the BAD and SET mindsets as a theoretical framework. A follow-on paper (Jones, Heffernan, & Albion, 2015) drills further into the D(istributed) of the BAD mindset and uses a distributive view of learning and knowledge as a theoretical framework. That distributive view of learning and knowledge is then used to analyse the experience of three teacher educators. The components of this view of learning and knowledge – situated, social, distributed, and protean – are then used to propose questions for further research. Research that could include DBR work that uses this view of learning and knowledge as a theoretical framework. Some other examples include:
- Special issue of the IRRODL journal titled “Connectivism: Design and Delivery of Social Networked Learning”
A collection of papers that use Connectivism (a prominent view of NGL) and other related frameworks.
- An interview with Laura Gogia talking about her work around Connected Learning (another view of NGL).
The page includes links to a range of supporting resources including Laura’s dissertation, including framing learning as connecting in which she connects a full range of prior education research and this PDF on connected course design.
The main value of a theoretical framework for your assignment is for you to focus on a specific understanding of NGL, what that understanding considers important, and thus how you should go about designing a context-specific solution to your problem. The idea is not that you develop yourself a detailed NGL framework or a framework as detailed as some of the examples above. However, the idea is that you should identify a framework that defines the specific view of NGL you are going to use.
Activity: What “views” of NGL are you aware of (e.g. connectivism, connected learning…)? Which of these resonate with your? Which appear appropriate to your problem and context? Why?
Reading: Read the section titled “Research questions” from Herrington et al (2007).
As your proposal won’t extend into a complete research project there is less of a need to spend a large amount of time formulating a research question. That said, my suggestion is that you still seek to formulate a question that gives a clear indication of what you hope to learn if you were to implement your proposal.
As an example, in this post some early questions are asked about how a combination of learning analytics and learner experience design might impact/improve the ability for teachers to “know they students”.
The work in Jones and Clark (2014) doesn’t have a specific research question, but the following might be appropriate
How can the reality/rhetoric chasm in institutional e-learning be closed?
In the following list, I’ve adapted Bakker’s (2014) list of criteria for a good research question to the context of this assignment, the criteria includes (there are other criteria listed, but these are the most relevant)
- It addresses a knowledge gap.i.e. your knowledge gap. You are unsure about how to use NGL to address some instructional problem that you have in your context.
- The question should be pragmatically and theoretical relevant.It should be something important and useful to you and your context. Better yet, it should be something of interest to others who are in similar contexts. (Don’t worry too much about the theoretical).
- Its main concepts are precise and anchored in the literature.The terms or phrases you use in the question should be used in ways that are commonly found in the literature and they should mean exactly what you want.e.g. adapting an example from this page you wouldn’t use the phrase “health information systems” in your research question if “decentralised health information systems” was a more precise/better fit.
- It should be manageable.i.e. something you can do. In this case, this is perhaps not a major problem as your proposal is focused on the planning, rather than the implementing and evaluating.Perhaps a more relevant way of looking at this is that what you plan is within the realms of you actually implementing it at some stage in the near future.
Bakker (2014) goes onto to suggest that there are two broad types of research questions in DBR
- Questions that ask about the characteristics of some intervention.
Examples provided by Bakker (2014) are
What are characteristics of valid and effective teaching and learning strategy to teach students about correlation and regression in such a way that they experience coherence between mathematics and the natural sciences? (Dierdorp, 2013)
What are the design characteristics for a practicable and effective learning, supervising and teaching strategy that enables VMBO students to recognize the functionality of biological knowledge of reproduction in work placement sites? (Mazereeuw, 2013)
- Questions that ask how
Examples provided by Bakker (2014) are
How can a learning-and-teaching strategy, aimed at the flexible use of biological concepts through recontextualising, be structured? (Wierdsma, 2012)
How can students be empowered in their opinion forming on personal and societal implications of genomics?
As suggested by Herrington et al (2007) the research question should arise from your problem and your “consultation with practitioners” and also be informed by your literature review (the next major section).
It is this basis on practical experience and basis in the literature that means that “can” questions such as “Can Masters students learning about NGL using NGL?” are not that interesting or relevant. The knowledge you’ve gained from the practitioners and your literature review should mean that you are fairly confident that your intervention will work. This is why DBR is more interested in “design” type of questions. i.e. How do you design a course about NGL using NGL? What are the characteristics of a good NGL course taught using NGL?
Activity: Start working on Section 1.4 The research questions from the Herrington template. As with all of the template “answer”, these are draft, answers you will continue to refine.
This is where your NGL theoretical framework can come in handy.
Reading: Read the section titled “Development of draft principles to guide the design of the intervention” from Herrington et al (2007).
These are meant to be some initial indicators, suggestions, guidelines etc for how you will go about thinking about developing your proposed intervention.
A design principle (fairly high level and in need of some refinement) you might extract from Jones and Clark (2014) would be
There would appear to be some significant benefit to exploring a dynamic and flexible interplay between the strategic and bricolage approaches to deciding what work gets done (n.p)
From Jones et al (2015) you might extract some principles that says institutional approaches to helping develop the TPACK of teaching staff should be
- Situated in the particular contexts in which learning and teaching occur.
i.e. not one websites or presentations that occur outside of learning and teaching.
- Social in nature.
- Distributed across individuals, other people, and tools.
- Recognise that digital technologies are there to be manipulated and modified.
I’ve generated those principles in a couple of minutes just now. Obviously for Assignment 2 you will spend much more time on these. Ensuring that they link to the problem, your literature review, and NGL.
The draft principles for your Assignment will come from the literature etc that you are drawing upon. Perhaps also from your peer review process.
Activity: Work through section 2.1 Draft principles in the literature in the Herrington DBR template. Also work through section 2.2 Technological affordances.
Reading: Read the section titled “Description of proposed intervention” from Herrington et al (2007).
Once you’ve done your literature review and extracted your draft principles, you can get into designing your intervention. What is it that you’ll change and how will you change it?
This recent blog post is a long-winded example of describing a particular intervention that is based on the principles for developing the TPACK of teaching staff mentioned in the last section. It is important how well the proposed intervention aligns with the principles and in turn aligns with the literature review that preceeded it.
Activity: Work through section 2.3 The design of the learning environment from the Herrington DBR template. In particular, note how you should be able to make direct connections between your design principles and the design of the learning environment.
Not a sequential process
The above has been presented as a sequential process.
- Identify the problem.
- Talk with people about the problem.
- Look at the literature and prior work to understand related work.
- Find a theoretical framework that can help guide design.
- Draft some design principles.
- Design an intervention based on those design principles.
Your experience will not be this neat of sequential. Completing assignment 2 will very iterative.
You more than likely already have an inkling of an idea of what you might like to try. That’s only human.
However, you should try – as Herrington et al (2007) mention – to avoid as much as possible starting with a specific intervention in mind. A better approach is to iterate back and forth between trying to answer the following questions.
- What is your problem/research question?
- What does the literature say?
- What are the draft principles?
- What is your intervention?
As you iterate back and forth between these phases, you should be actively trying to ensure that they align. i.e. that your intervention is actually informed by your principles, which is in turn drawn from literature, and is obviously linked to the nature of the problem you are trying to solve.
As you read more, think more and get more feedback your answers to the above questions will evolve and change. Go with the flow. This to and fro between these questions will probably describe much of what you’ll be doing over the coming weeks.
But do make sure that they align in your final proposal.
Where to now?
In short, start your proposal.
You will almost certainly have questions and want more clarification. Ask away.
Design-based research is an increasingly common approach and there are many online resources that can provide additional pointers. For example, you might like to check out my Diigo resources tagged with “dbr” or you might like to check out some of the Mendeley groups on design-based research.
Bakker, A. (2014). Research Questions in Design-Based Research (pp. 1–6). Retrieved from http://www.fi.uu.nl/en/summerschool/docs2014/design_research_michiel/Research Questions in DesignBasedResearch2014-08-26.pdf
McKenney, S., & Reeves, T. C. (2013). Systematic Review of Design-Based Research Progress: Is a Little Knowledge a Dangerous Thing? Educational Researcher, 42(2), 97–100. doi:10.3102/0013189X12463781
Jones, D., & Clark, D. (2014). Breaking BAD to bridge the reality/rhetoric chasm. To appear in the Proceedings of ASCILITE’2014. Dunedin, NZ.
Jones, D., Heffernan, A., & Albion, P. R. (2015). TPACK as shared practice: Toward a research agenda. In D. Slykhuis & G. Marks (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2015 (pp. 3287–3294). Las Vegas, NV: AACE.