Archive for the ‘Jem Symposium Heerlen’ Category

MILCA - A Mobile and Interactive Learning Environment on Campus

Friday, September 26th, 2008

Presentation by Kin Choon Yow (Nanyang Technological University, Singapore) at the 1st World Summit of the Open Knowledge Society, Athens, 24-26 September 2008. Track: Knowledge, Learning, Education, Learning Technologies, and eLearning for the Knowledge Society.

Current learning scenarios are not interactive. But in particular Asian students are very shy and do not dare to ask questions in the classroom. But they are more likely to engage in technologies and post their question online. So can we make use of mobile classroom scenarios? The presented system is build on two core technologies: Multimedia Messaging Service (MMS) and Live Audio Streaming. Students can follow the lecture anywhere on campus. The system is also used in classroom and particularly helpful for lecture with high numbers of participants. Students can send their questions via MMS to the lecturer, MMS questions are displayed underneath the slides, and lecturers can immediately react (real-time lecture feedback).

Evaluation Results: Students like it (some find it cool). Shy students can build up confidence to ask questions. Allow students asking questions even in large classes. Lecturers can understand better how students think and if/how they follow the course. Lecturer can adjust teaching pace immediately.

Questions: How to deal with high numbers of questions during the class? Currently, lecturer answer questions at the end of each sub-section. Some question are similar and are skipped. Do students loose much (attention) time when typing the messages? But the young generation is pretty fast. What are challenges of distance-teaching? Teacher seems to have problems to talk to a screen for several hours. Teacher loose the control and immediate feedback when teaching online, they have bigger influence in the classroom. The social experience of regularly going to university might actually be an important aspect of the learning experience.

Typicality of exercises

Saturday, September 20th, 2008

Discussion at the JEM Symposium on “math tutoring: tools and feedback” in Heerlen, Netherlands.

I have been reading a two papers by Manfred Kerber, Erica Melis, and Jörg Siekman on the typicality of examples which raised my interest and which I would like to further elaborate on. See below my notes on both papers. During the JEM Symposium I tried to get other perspectives on this topic:

Johan Jeuring mentioned that their worked out examples (in their exercise web service) are actually (hopefully/ ideally/ potentially) typical examples to illustrate a problem. The study of Gemma Corbalan showed that this kind of feedback was more helpful then only providing feedback to a concrete solution.

However, Johan was wondering whether the context-aware handling of typical exercises would provide a significant variance, that is, if the users’ subject views on what is typical really varies enough that a context-aware exercise framework and respective markup makes sense (in particular for mathematics; analogously to our notation framework). However, he also mentioned that in particular in computer science there are a lot of changes going on e.g. wrt. to which programming language is used for illustrating specific programming concepts.

George Goguadze said that he would not base the selection of examples on the typicality. First of all, teachers have a subjective view on which examples/ exercise are typical for a particular audience. And secondly, students have a subjective understanding what is typical (e.g. base don their prior education). But his comment actually illustrates that their are different understandings of typicality, so explicating these might potentially be helpful. For example, teachers can provide typical examples in their lecture, but receive feedback on the student’s view on these. Based on the exercise material (and a pool of further exercises) of the teacher, a tools could select the typical exercises from a student’s perspective (based on the interrelation and annotation of exercises).

Notes on the two papers by Manfred Kerber etAl:

  • Typical examples are examples that are representative for a particular situation or concept; they play an important role in human knowledge representation and reasoning (see also reasoning by analogy, case based reasoning)
  • Typical examples are easy to remember and are sufficient to catch important aspects of the general case.
  • We can distinguish different types of typical examples (depending of the type of concepts or situation they help to explain): procedural examples (e.g. illustrating an induction proof), typical examples for concepts (e.g. a violin, a chair), typical examples of a situation (e.g. opening a door); typical examples for a proof or a theorem, typical examples for a plan
  • Typicality of examples is a well-known research area in empirical psychology (see references below)
  • Representation of examples by a data structure like a semantic net, frame structure, or within neural net.
  • Typicality ratings distinguishes typical examples from atypical examples: direct ratings (Is this a typical example?); reaction time (Answer true or false: [example] is a [category-name]); reproduction of examples (list or draw an example of a category member); asymmetry of similarity rating (less typical examples are often considered to be more similar to typical example than the converse); generalization (Humans generalize more likely from typical examples than from arbitrary examples)
  • CoPs and typicality: Can typicality be influenced by context parameters? Do interpretation (preference) vary among different communities of practice? Can context-aware handling of typical example help to find and select typical example for a CoP or an individual? Do the typical examples (identified for a specific context/ CoP) improve understanding (in contrast to the default typical example)?
  • CoPs and typicality: Are some notations typical for a CoP (e.g. if their are most frequently used by this CoP or preferred by most members of the Cop)?

Further References:

Interactive Exercises in ActiveMath: Cross-Domain Diagnosis and Automated Tutorial Strategies

Friday, September 19th, 2008

Presentation by George Goguadze on his Exercise Module in the ActiveMath system at the JEM Symposium on “math tutoring: tools and feedback” in Heerlen, Netherlands.

Features
A key feature of ActiveMath is a user model used for adaptation (e.g. generation of individualized courses) and guidance of the user. The system and various materials are available in various languages. Users can request further information/ exercise on the course material (self-regulated learning). Student can browse an ontology for the course (concept mapping tool), semantic search, students can assembler their own course material from different sources and the system finds exercises/ examples automatically for the selected concepts (assembly tool), function plotter, CAS console, teacher tool (visualization of students’ performance) and student inspector.

Further thoughts on George Goguadze’s Exercise Module
George Goguadze provides an XML-based markup of exercises, in particular, authors can mark up every step, i.e. a status of the exercise solution graph, and can attach feedback to the edges of this graph. The authors markup is combined with the system’s internal exercise strategies and provides an interactive experience of the user.

The exercise strategies cannot be created or modified by the author of exercises (in contrast to the approach by Johan Jeuring), but are created and maintained by the developer of the system. Technically, strategies are created by inheriting from an abstract strategy (Java) class and modifying it respectively. George mentioned that he covers various strategies and that, if steps are not defined, he provides either a dynamic strategy that adapts to the user or a fall back (a default interaction step). However, he is aiming towards a declarative language for these strategies, consequently, the approach of Johan Jeuring is very interesting to him. He will be analysing their approach carefully and, potentially, integrate their web service into the exercise module of ActiveMath.

George pointed us to his recent papers on exercises and will update the lists soon. He can also provide examples, which can help us to understand his approach. Moreover, the source code of the ActiveMath system including his exercise module is open source and available for download at the ActiveMath website.

George reminded us to not do redundant work. We will do our best to avoid this and want to carefully verify whether this approach can be applied to our types of problems: Our problems are currently marked up differently (using OMDoc 1.2). The ActiveMath content seems to be based on OMDoc 2000; while the markup of exercises provides an non-standardized extension of the OMDoc format. We have to verify if and how these extension can be integrated in OMDoc 2.0, e.g. by adding an OMDoc exercise module as extension to the current OMDoc quiz module. For our existing pool of problems, we have not yet considered a markup of single steps, this can be a potential extension of our problem markup. However, we also want to classify and interlink problems. George mentioned that he is also doing this. Each interaction step can be annotated with a certain competencies. A competencies always refers to a concept (a symbol, theorem, …). These competencies can be reached by the user by solving exercises. In the user model he then stores the conceptID, the competencies and further metadata. These competencies on the first level of the user model are used to compute the users master level on the second layer (his overall competencies in a concept) and which are futher computed to the user’s average competencies (or knowledge?). It is not yet clear to me, whether and how exercises are classified in the exercise module and based on which criteria they can be selected. As far as I know, it is already possible to consider certain difficulties: For example, if a student fails an exercise he can be provided with a more simpler one (simple with respect to the competencies in his user model, as this is a subjective property).

Moreover, our problems are targeted to Computer Scientists. George mentioned that he can already handle some CS problems, such as automata exercises. We want to look at his existing examples as well as at our existing problem pool. Johan Jeuring also mentioned that he will be working on supporting interactive programming exercises, which can be quite interesting for us and maybe also ActiveMath.

Towards Open Content for Mathematics: Starting from the Math problem

Friday, September 19th, 2008

Presentation by Leendert van Gastel (AMSTEL UvA) at the JEM Symposium on “math tutoring: tools and feedback” in Heerlen, Netherlands.

How can we keep research products alive after the project is over? Towards a business model for research projects.

Further Readings

Interactive exercise in DME

Friday, September 19th, 2008

Presentation by Peter Boon (Freudenthal Institute for Science and Mathematics) on his Digital Math Environment (DME) at the JEM Symposium on “math tutoring: tools and feedback” in Heerlen, Netherlands.

Focus on Realistic Mathematics Education (RME): Distinction of different level of formalization: (1) meaningful situation and problems, (2) models and representations and (3) formal mathematical language.

Feedback types:

  • explicit vs. implicit feedback
  • text based feedback vs. representational feedback
  • direct vs. delayed feedback
  • intelligent vs. functional feedback

Examples on the DME Website.

  • Advanced equation module: visualization for correct and incorrect solutions as well as tooltip with immediate feedback.

The Leibzig autotool eLearning, eTesting system for student problems in Theoretical Computer Science

Thursday, September 18th, 2008

Presentation by Johannes Waldmann (HTWK Leipzig) on his math tutoring system autotool at the JEM Symposium on “math tutoring: tools and feedback” in Heerlen, Netherlands.

Problems are marked as

  • demo (too easy; for illustration only)
  • mandatory (must submit at least one correct solution before deadline, any number of attempts with immediate feedback)
  • optional (too hard, prize question)

Students can also solve problems after the deadline, these are graded but not counted. But these solutions can be saved and used for exams preparations.

Problem areas:

  • formal language: grammars, regular expressions
  • automata, models of computation
  • graphs (using graphviz for visualization)
  • logic

Topics of his course

  • introduction to algorithms, sorting: sorting networks (discuss: specification, correctness, lower bounds)
  • introduction to complexity (search problems): COL (NP), Lunar Lockout (PSPACE), PCP (RE)
  • introduction to programming: simple (imperative): Collatz Sequence (/Inverse questions), types (syntax and semantics of expressions, type checking)
  • data structures

Problem types:

  • certificate: find a small object with property …
  • forward: what is the result of the input
  • backward: what is the input if the result is
  • holes: fill in missing steps such that input … gives result …

Johannes Waldmann suggested to use autotool for about half the exercises of a course and the other half for formal proofs (university) or programming (university of applied science). Students like the tool and appreciate the immediate response. It is also used for competitions. Autotool is implemented in haskel, easy to use and to install, and supports a first multilingual exercises.

Using feedback tools, some ideas on the didactics

Thursday, September 18th, 2008

Presentation by Josje Lodder (Open Universiteit Nederland) on different feedback studies at the JEM Symposium on “math tutoring: tools and feedback” in Heerlen, Netherlands.

Not so many tools on linear algebra, e.g. the row operation calculator in the linear algebra toolkit.

Why are we using these tools: gap between high school and university, extra draining during courses, save time to teach concepts (practice can be done at home), preparation for re-examination, distance education.

What results do we expect: learning of algorithmic skills, efficient solution strategies, …

condition: bug free, user friendly, in accordance with course, offering help, providing feedback, different notations confuse the students (tools should adapt to the course material)

Call for improving math tutoring tools:
The presentation by Josje Lodder allowed a very nice summary on the scope of the symposium. She pointed to the reason why we want math tuoring tools and what results we expect when using them in teaching and pointed out conditions these tools should satisfy. Many of these condition still seems to be an issue. This symposium has gathered a very good collection of existing work aiming at providing appropriate feedback to the students while solving a problem, which seems an important condition of tutoring systems. This includes basic technologies and services, interfaces (partly making use of these services), and evaluations. Most of the participants of the symposium strongly collaborate; much effort is invested to integrate services and systems: Just two point out two example: The web services presented by Johan Jeuring is used by the MathDox system, which material is now available in Moodle; while the MathDox is editor is used in the ActiveMath system. It is very good to see these joint efforts towards making math tutoring more usable and efficient.

Learning and motivational effects of different forms of feedback in linear algebra problems

Thursday, September 18th, 2008

Presentation by Gemma Corbalan (Open University of the Netherlands) on different feedback studies at the JEM Symposium on “math tutoring: tools and feedback” in Heerlen, Netherlands.

Study 1: Exploring the students’ perceptions regarding three different types of feedback generated by the MathDox system (9 students 1st year): Feedback on the final solution (correcting the answer), worked out examples (full solution of the problem), step-by-step.

Some Questions of the Questionnaires:

  • This type of feedback was informative?
  • The content of this type of feedback made the problem easier to solve.
  • Describe what you like the most about this type of feedback?
  • Which of te three types of feedback do you prefer the most?
  • Which of the three feedback types was less informative?
  • Selecting the feedback given an easy, medium, and difficult problem

Results: Students preferences of feedbacks types e.g.

  • feedback on problem-solving was preferred, in particular, step-by-step instructions
  • more detailed feedback with increasing difficulty

Study 2: To measure the effectiveness, efficiency and motivational effects of two types of feedback generated by the MathDox system
during practice

  • Measuring the mental effort: by asking students to indicate their mental effort based on a scale; after each exercise (student provided with the worked out examples or the step-by-step solving invested less mental effort)
  • Measuring the performance and time

after practice: questionnaire of study 1

post practice:

  • Near/ far transfer problems were given (to check if the quality of learning increased)
  • Asking for the mental effort for solving the near/ far problems
  • Efficiency (combining the performance and mental effort).

As far as I understand, this study computes the efficiency in solving a problem based on the mental effort and the final score. However, when copying a solution for a problem, I have a minimal mental effort and (if the copied solution is correct) a maximal score, thus leading to a maximal efficiency. Which is true for the task of solving a problem. However, I am wondering whether this also improves the learning of a student. But I assume this is why the “far transfer problem” were provided. And in the study, the students provided with step-by-step instructions and worked out example actually scored better in these far transfer tests. They also invested less mental efforts and thus were more efficient.

Further plans are studies on the effects of different types of feedback adapted to the difficulty of the problems.

Study 3: Comparing the LA system and MathDox
Mental effort was rather low with both systems. Both groups had similar post problem results. The general perception of usability was rather high in both systems. Learners were positive towards the eLearning experience.

Specifying Strategies for Exercises

Thursday, September 18th, 2008

Presentation by Johan Jeuring (Open Universiteit Nederland) at the JEM Symposium on “math tutoring: tools and feedback” in Heerlen, Netherlands.

Johan Jeuring presented his work on exercise strategies presented at the MKM conference. The presentation was followed by three talks on feedback tools that make use of the exercise web service (by Bastiaan Heeren, Hans Cuypers (MathDox), and Rick van der Meiden)

For me the introduction of Johan’s talk was extremely interesting as it reflects scientific practice for solving problems in different scientific areas. It would also be interesting to look at the different types of strategies that the project has identified as well as concrete instances of these strategies for concrete exercises (e.g. created for the feedback tools).

A web service is available. Only for own personal testing purpose a GUI has been implemented. A potential system using the web service should interpret/ wrap the feedback of the service. Users are not intended to directly use it. For example in the GUI, the numbers of steps are displayed and the user’s progress. Users can solve more then one step with one interaction, thus the progress and feedback should be wrapped by a system using this web service.

The flexibility of the problem solving depends on the strategies. Very strict strategies reduce the number of possible paths; non-specified steps cause complaints. Again, the feedback needs to be re-interpreted for the end-user.

Following Johan’s talk, Bastiaan Heeren illustrated the interaction with the web service, showing some very nice examples.

Hans Cuypers introduced the services of the MathDox system. Among others he mentioned that back engines such as mathematics return typical notations and solutions; here typicality refers to the specific syntax/ characteristics of the back end service. That’s an interesting point for us in the discussion on typical exercises and examples, as the concrete system can be a potential context parameter for typicality. The MathDox material has been integrated intof Moodle (a SCORM package has been created for the import) and will be used next semester.

Rick van der Meiden (Technische Universiteit Delft) presented the direct feedback in an eLearning system for linear algebra.

Further Details

Procedural Skills in different scientific areas

  • Mathematics: calculate the value of an expression
  • Computer Science:
  • Physics
  • Biology

Tutoring Systems for procedural skills (providing feedback)

  • Wisweb (Freudenthal Institute)
  • LeActiveMath
  • MathXPert
  • Aplusix

Strategies types:

  • Expert Strategy
  • Arithmetic Strategy
  • ….

Use of Strategies for error diagnosis and feedback; embedding in other systems.

  • Strategy unfolding
  • hints, feedback, …

Further Readings

OKUSON, online homework excercises with mathematical content

Thursday, September 18th, 2008

Presentation by Frank Luebeck (Lehrstul D for Mathematics, RWTH Aachen) at the JEM Symposium on “math tutoring: tools and feedback” in Heerlen, Netherlands.

Due to huge number of students, the university wanted to switch from classical exercises to online exercises, partially graded automatically (multiple choice and similar).

Requirements

  • Exercises are specified in (standard) LaTeX; no additional (semantic) macros.
  • Individual sheets to avoid copying of solutions!
  • High quality display on the web; but they use images only, which reduces/ hampers the accessibility e.g. for visual impaired people or screen readers.
  • …more requirements …

Implementation

  • Existing Systems (WebAssign, WebCT, Moddle) has a lot of functionality they didn’t need, and fulfilled few of their requirements (in 2002)
  • OKUSON is home-made and thus fullfills all requirements; programmed in python; GPL license; hobby project so no technical support; buildin webserver with on-the-fly validation of XHTML; basically a webpage with can be easily extended
  • exercise sheets are in simply XML-based file formats (see exercise template for an example; points to either classical exercise sheets in LaTeX or to interactive exercises); exercises are created with an offline texteditor
  • Used by all mathematical freshmen lectures at RWTH Aachen, several German Universities (Hamburg, Hannover, …), used in England and Irland; many users; few bug report and support questions
  • Implementation was very easy, the most difficult part was the invention of exercises

Student features

  • register for system
  • Download exercise sheet in PDF; enter solutions into the XHTML. XHTML is created by generating images for each exercise snippets from the LaTeX and inserting these images in an XHTML form. Student can solve the exercise several time until the deadline.
  • Submit solutions, see grading, register for exams

Further details:

  • question type: Single choice, multiple choice, answer as text (numbers and text)
  • grading: +1 point for correct answer, -1 point for wrong answer, 0 points for no answer
  • Interactive questions with variants (alternative question texts) with concrete instances; these alternatives are used to generate individual exercise sheets (based on the student ID); problem: the degree of difficulty might vary among these alternatives (e.g. template)
  • statistics: how many percent successfully solved a variant of an exercise sheet; on-the-fly visualization of the statistics …
  • all data stored in text files (also user data and results); simply querying of this data and further UNIX computations (grep, sort, …)

Evaluation:

  • Did these online question help to understand linear algebra?
  • Students very much liked the online exercises, were more motivated. Online system was used to approve the students registration for the pre-diploma. Grades were equal to results in traditional exercises; good students remained good; poor students remained poor. Participation increased (number of students taking the qualification exams increased). No complaints during the change from traditional to online exercises. Feedback was given during the lectures.

Future work: System can be easily be integrated with e.g. GAP or other system for automatic verification and immediate feedback.

Project Webpage